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EXECUTIVE  SUMMARY 


1.0  Introduction 

This  summary  is  designed  to  serve  as  a  stand-alone  document  as  well  as  part  of  this  report.  For 
that  reason,  the  reader  will  find  some  duplication  of  verbiage  and  figures  between  the  summary 
and  the  full  report.  The  Naval  Air  Warfare  Center  Weapons  Division  (NAWCWPNS)  was  a 
committed  and  effective  partner  for  the  Joint  Advanced  Distributed  Simulation  Joint  Test  and 
Evaluation  (JADS  JT&E)  Joint  Test  Force  (JTF)  in  the  planning,  preparation,  and  execution  of 
the  Linked  Simulators  Phase  (LSP)  of  the  System  Integration  Test  (SIT). 

2.0  JADS  Overview 

The  JADS  JT&E  was  chartered  by  the  Deputy  Director,  Test,  Systems  Engineering  and 
Evaluation  (Test  and  Evaluation),  Office  of  the  Under  Secretary  of  Defense  (Acquisition  and 
Technology)  in  October  1994  to  investigate  the  utility  of  Advanced  Distributed  Simulation  (ADS) 
technologies  for  support  of  Developmental  Test  and  Evaluation  (DT&E)  and  Operational  Test 
and  Evaluation  (OT&E).  The  program  is  Air  Force  led,  with  Army  and  Navy  participation.  The 
JTF  manning  includes  23  Air  Force,  13  Army,  and  2  Navy.  Science  Applications  International 
Corporation  and  Georgia  Tech  Research  Institute  provide  contracted  technical  support.  The 
program  is  nominally  scheduled  for  five  years. 

The  JADS  JT&E  is  directly  investigating  ADS  applications  in  three  slices  of  the  T&E  spectrum:  a 
System  Integration  Test  (SIT)  which  explores  ADS  support  of  air-to-air  missile  testing,  an  End- 
To-End  (ETE)  test  which  explores  ADS  support  for  Command,  Control,  Communications, 
Computers,  and  Intelligence  (C4I)  testing,  and  an  Electronic  Warfare  (EW)  test  which  explores 
ADS  support  for  EW  testing.  The  JTF  is  also  chartered  to  observe,  or  participate  at  a  modest 
level,  in  ADS  activities  sponsored  and  conducted  by  other  agencies  in  an  effort  to  broaden 
conclusions  developed  in  the  three  dedicated  test  areas. 

Phase  1,  the  Linked  Simulators  Phase  (LSP),  of  the  SIT  is  the  subject  of  this  summary  report. 

3.0  SIT  Overview 

The  SIT  is  a  two  phase  test  designed  to  examine  the  application  of  ADS  technology  in  two 
architectures.  The  first  phase  employed  an  all  simulator  architecture  which  incorporated  a 
manned  F-18  avionics  lab  (simulator)  at  China  Lake  NAS  as  a  shooter,  a  manned  F-14  avionics 
lab  (simulator)  at  Point  Mugu  as  a  target,  and  a  missile  hardware-in-the-loop  (HWIL)  simulation 
lab  (SIMLAB)  at  China  Lake  which  generated  AIM-9  missile  flyouts  and  injected 
countermeasures  (flares).  The  second  phase  of  the  SIT  employs  an  architecture  which 
incorporates  a  live  F-16  shooter  aircraft,  a  five  F-16  target  aircraft,  and  an  Advanced  Medium 
Range  Air-to-Air  Missile  (AMRAAM)  HWIL  simulation  hosted  in  the  Eglin  AFB  Guided 
Weapons  Evaluation  Facility  (GWEF).  This  document  summarizes  the  LSP  activities. 
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4.0  LSP  Test  Plan  Overview 


4.1  Purpose 

The  LSP  was  designed  to  examine  the  real-time  interactions  between  networked  manned  flight 
simulators,  a  missile  HWIL  simulation  facility,  and  a  control  center.  The  focus  of  the  examination 
was  on  the  relationships  between  network  performance,  system  under  test  (SUT)  data,  and  test 
measures  of  interest.  In  general  terms,  the  purpose  was  to  collect  data  on  the  quality  and  usability 
of  test  data  in  this  particular  distributed  test  architecture.  The  test  objectives  were: 

4.1.1  Objective  1 :  Assess  the  validity  of  AIM-9  data  obtained  in  the  LSP  ADS  configuration 

4.1.2  Objective  2:  Assess  utility  of  LSP  ADS  configuration  for  parametric  studies 

4.1.3  Objective  3:  Assess  effect  of  latency  on  validity  of  test  results 

4.1.4  Objective  4:  Assess  ability  of  LSP  ADS  configuration  to  support  AIM-9  testing 
(This  test  objective  was  broken  into  subobjectives  as  follows.) 

4. 1 .4. 1  Subobjective  4-1 :  Assess  capability  of  network  to  provide  required  bandwidth  and 
connectivity 

4. 1.4.2  Subobjective  4-2:  Assess  the  effects  of  ADS-induced  errors  on  LSP  test  results 
validity 

4. 1 .4.3  Subobjective  4-3:  Assess  adequacy  of  standard  data  protocols  for  LSP  test 

4. 1.4.4  Subobjective  44:  Assess  reliability,  availability,  and  maintainability  of  ADS 
network 

4. 1 .4.5  Subobjective  4-5:  Assess  capability  for  centralized  test  control  and  monitoring 

4.2  Approach 

The  F/A-18  Weapon  System  Support  Facility  (WSSF)  at  China  Lake  and  the  F-14D  Weapon 
System  Integration  Center  (WSIC)  at  Point  Mugu  were  the  shooter  and  target,  respectively.  The 
shooter  “fired”  the  AIM-9  in  the  SIMLAB  at  the  target  which  could  respond  with 
countermeasures.  Runs  were  controlled  from  a  test  control  center  which  ensured  all  nodes  were 
ready  for  each  run,  issued  start/stop  directions,  and  processed  data  packets  for  real  time  analysis 
of  system  performance.  Test  control  was  exercised  from  the  Battle  Management  Interoperability 
Center  (BMIC)  at  Point  Mugu  while  the  JADS  Joint  Test  Force  was  physically  relocating. 
Control  switched  to  the  JADS  Test  Control  and  Analysis  Center  (TCAC)  after  their  move  was 
complete. 

Information  was  exchanged  between  participants  in  the  form  of  Distributed  Interactive  Simulation 
(DIS)  Protocol  Data  Units  (PDUs).  Entity  state  data  (positions,  velocities,  accelerations, 
attitudes,  attitude  rates)  at  the  output  node  were  converted  from  simulator  format  to  PDUs,  and 
reconverted  at  the  receiving  end,  into  simulator  format.  An  exception  was  the  link  between  the 
Stores  Management  System  of  the  shooter,  and  the  missile  in  the  SIMLAB  which  used  1553  data 
bus  format.  The  architecture  is  shown  below. 
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Figure  ES-1.  Linked  Simulators  Phase  Test  Configuration 

A  baseline  engagement  profile  (LPN-15)  was  selected  from  the  AIM-9M-8/9  Joint  Initial 
Operational  Test  and  Evaluation  (JIOT&E)  test  series  (conducted  17  May  1993  to  29  October 
1993). 


This  single  engagement  geometry  was  the  basis  for  all  trials  in  the  LSP.  The  selection  of  this 
baseline  from  the  16  live  fire  profiles  of  the  AIM-9M-8/9  JIOT&E  was  based  on  three  factors: 
(1)  the  shooter  was  an  F/A-18,  (2)  flares  were  deployed,  and  (3)  sufficient  five  fire  data  were 
available  for  V&V  of  the  LSP  trials.  Additional  details  on  LPN-15  are  in  Appendix  A.  The 
profile  is  depicted  in  Figure  ES-2. 
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•  10,400  ft  /  0.72  mach 

•  58  0  angle  off  tail 

•  3.6  g  level  turn 

•  flare  countermeasures 


F/A-18C 

•  11,300  ft/ 0.71  mach 

•  0  °  angle  off  boresight 


Figure  ES-2.  Live  Fire  Profile  (LPN-15, 9  June  93) 

The  test  plan  scheduled  three  blocks  of  simulation  “missions”  as  shown  in  Figure  ES-3.  The  first 
to  do  Verification  and  Validation  (V&V),  the  second  to  do  parametric  studies,  and  the  third  to 
study  latency. 
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Figure  ES-3.  Linked  Simulators  Phase  Test  Schedule 


The  planned  schedule  is  shown  on  the  upper  half  of  the  Figure  ES-3. 


4.3  Instrumentation 


One  of  the  major  distinguishing  characteristics  of  a  distributed  testing  architecture,  as  opposed  to 
a  distributed  training  architecture,  is  in  the  degree  of  instrumentation.  The  LSP  architecture  was 
instrumented  to  a  level  which  supported  the  measurement  of  latency  values,  synchronization 
differences,  position  transformation  errors,  missile  seeker  performance,  and  the  whole  gamut  of 
network  performance  measures. 

5.0  LSP  Test  Results 

5.1  Schedule 

If  the  reader  will  refer  to  the  bottom  half  of  Figure  ES-3,  he  will  fmd  the  time  lines  for  actual 
testing.  The  time  required  to  integrate  and  check  out  the  architecture  was  significantly  longer  than 
planned.  Full-up  architecture  configurations  were  required  to  verify  fixes  to  integration  problems. 
(Stand-alone  fixes  were  postulated,  and  tried,  but  most  fell  short  when  the  network  was 
reinitiated.)  The  periods  of  integrated  verification  work  were  called  “lab  periods”,  and  they 
moved  scheduled  test  activity  to  the  right  on  the  timeline.  In  fact,  there  were  still  some  problems 
at  the  start  of  execution  of  the  V&V  missions,  and  one  could  argue  that  we  weren’t  really  ready 
to  test  until  the  Nov  test.  The  3  to  4  month  slip  was  a  reflection  of  number  of  things: 

1 .  This  technology  is  definitely  not  “plug-and-play”  in  T&E  applications. 

2.  There  is  a  significant  learning  curve  for  people  inexperienced  in  establishing  these 
architectures. 

3.  The  learning  curve  applies  at  every  node  of  a  distributed  architecture. 

4.  System  integration  is  the  name  of  the  game. 

If  the  same  team  of  people  were  to  construct  a  similar  architecture  now,  set-up  and  check  out 
would  probably  take  a  small  fraction  of  the  time  experienced  on  this  first  test  sequence. 

5.2  V&V  Missions 

The  underlying  assumption  in  the  approach  to  V&V  was  that  JADS  would  quantitatively  compare 
the  representation  of  participant  and  SUT  behavior  in  the  LSP  architecture  with  the  results  of  the 
five  test.  The  statistical  reality  was  that  we  only  had  a  single  data  point  on  the  live  side  of  the 
comparison,  and  a  rigorous  quantified  approach  was  not  practical.  The  test  force  fell  back  upon  a 
qualitative  comparison  of  live  profiles  with  the  ADS  profiles,  and  found  that  comparison  useful. 

5.3  Parametric  Missions 

The  expectation  for  the  parametric  studies  was  that  a  selected  set  of  “best”  runs,  recorded  during 
ADS  testing,  would  be  replayed  in  an  automatic  mode  while  selected  parameters  were  varied  in  a 
controlled  fashion.  In  fact,  since  the  aircraft  simulators  were  designed  to  operate  with  a  human  at 
the  controls,  there  was  no  way  to  implement  automatic  runs.  The  extensive  parametric  activity 
planned  for  was  scaled  back.  However,  the  pilots  were  able  to  achieve  reasonably  tight  adherence 
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to  scripted  profiles  manually,  and  the  test  force  concluded  that  the  ADS  architecture  would  be 
useful  for  parametric  studies. 

5.4  Latency  Missions 

The  schedule  slippage  discussed  in  Par.  1.4.1  precluded  execution  of  the  Latency  Study  Block  of 
LSP  as  originally  intended.  Nonetheless,  the  JTF  collected  sufficient  data  to  support  a  thorough 
understanding  of  latency,  and  its  impact  on  test  data  in  this  particular  architecture.  Latency  was 
addressed  in  two  components,  transmission  latency,  and  processing  latency.  Transmission  latency 
is  essentially  the  time  associated  with  data  travel  through  the  WAN  legs.  That  component  of 
latency  was  predictable  and  statistically  well-behaved.  Obviously,  transmission  latency  is  a 
function  of  distance,  but  for  this  test  the  average  latency  on  the  140  mile  leg  was  about  20  ms. 
Early  on,  processing  latencies  could  be  as  high  as  several  hundred  ms.  Compared  to  transmission 
latencies,  processing  latencies  were  not  as  statistically  well-behaved.  In  the  later  test  events,  while 
it  varied  from  leg  to  leg,  processing  latency  was  about  twice  the  magnitude  of  the  transmission 
latency  component. 

5.5  General 

5.5.1  Reliability 

The  reliability  of  the  long-haul  elements  of  the  network  (the  Wide  Area  Network  (WAN)  was 
very  good.  The  availability  of  the  complete  LSP  ADS  configuration  was  on  the  order  of  85%. 
Reliability  expressed  in  terms  of  successfully  linked  runs  was  about  67%  over  the  entire  span  of 
rehearsal  and  test  activity.  Rims  failed  for  a  wide  variety  of  reasons  involving  people,  computers, 
instrumentation,  data  recording  equipment,  and  simulator  performance. 

5.5.2  ADS-Induced  Errors 

ADS-induced  errors,  aside  from  latency,  were  tolerable.  Position  transform  errors  were  on  the 
order  of  two  to  three  feet.  Initially,  there  was  a  significant  position  divergence  problem  between 
the  target  position  as  determined  by  the  SIMLAB  and  the  true  target  location.  This  was  caused 
by  the  manner  in  which  the  SIMLAB  simulation  processed  the  incoming  target  data  for 
presentation  to  the  seeker  and  by  coordinate  transformation  inaccuracies.  The  errors  were 
reduced  by  nearly  an  order  of  magnitude  when  the  transformation  was  corrected  and  the  PDU 
update  rate  increased.  The  remaining  error  of  about  30  ft  is  still  significant,  but  the  problem 
appears  solvable. 

5.5.3  Test  Control 

Test  control  procedures  were  refined  throughout  the  preparation  process,  and  worked  well  during 
testing. 
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5.6  Fulfillment  of  Test  Objectives 
All  test  objectives  (See  par.  4.1)  were  met. 

6.0  Lessons  Learned 

6.1  Technical 

The  test  team,  and  the  system  experts  from  each  node,  need  ready  access  to  experts  in  data 
transformation  and  interface  software. 

Synchronization  of  activity  in  a  network  is  difficult.  Not  every  processing  element  in  a  distributed 
architecture  has  direct  access  to  GPS  time,  so  a  “master  clock”  has  to  be  built  into  the 
architecture. 

Commonality  of  ADS  hardware  and  software  is  necessary.  Routers  from  different  vendors  may, 
or  may  not,  interoperate  efficiently.  Supposedly  interoperable  software  may  not  work  as 
advertized  —  carefully  review  version  numbers. 

Understand  before  designing  a  distributed  architecture  what  the  latency  requirements  are. 
Incrementally  build  to  satisfy  them. 

Special  test  instrumentation  and  tools  are  required  to  support  distributed  T&E.  The  tool  set 
must  support  rapid  identification  and  characterization  of  network  problems  and  time  tagging  of 
data  elements  sufficient  to  support  analyses. 

6.2  Infrastructure  and  Process 

ADS  requirements  must  be  developed  early,  understood  by  all  parties,  and  thoroughly 
documented.  The  communications  required  to  exercise  test  control  must  be  identified  early. 

SUT  experts  must  be  involved  from  the  outset. 

The  architecture  build-up  must  be  incremental,  beginning  with  check  out  of  the  ADS  elements  in  a 
stand-alone  mode,  and  evolving,  step  by  step,  to  the  fully  integrated  configuration. 

Problem  solving/fixes  frequently  require  verification  in  a  full-up  network  environment. 

Detailed  planning  for  data  management  is  a  necessary  precursor  to  testing. 

Contracting  to  support  ADS  should  most  often  be  on  a  cost  plus  basis.  There  are  too  many 
unknowns  to  make  fixed  price  contracting  a  viable  option. 

Centralized  test  control  processes  have  to  be  integrated  with  established  local  processes  and 
procedures. 
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Configuration  control  in  a  distributed  architecture  is  difficult,  but  essential.  The  test  organization 
needs  to  be  represented  at  each  test  architecture  node. 

7.0  Conclusions 

This  particular  test  architecture  would  not  support  valid'  closed-loop  interactions  between  the 
missile  and  target.  It  could  support  closed-loop  interactions  between  the  shooter  and  the  target 
for  such  purposes  as  test  rehearsal.  A  similar  architecture  with  more  representative  (realistic) 
flight  simulators  could  probably  support  tactics  development  and  training. 

With  more  attention,  and  more  time,  better  NIUs  could  have  been  developed,  and  better  NIUs 
might  improve  accuracies  and  latencies,  and  ease  synchronization  difficulties. 

Preparation,  set  up,  calibration,  and  check-out  activities  are  more  challenging  and  time  consuming 
in  a  distributed  environment.  The  time  lines  from  this  particular  test,  however,  should  be  viewed 
with  caution.  The  technology’s  use  in  T&E  is  still  in  its  infancy,  and  the  test  agencies  involved  in 
the  integration  activities  are  climbing  a  steep  learning  curve.  If  another  tester  were  to  work  with 
China  Lake  and  Pt.  Mugu,  we  believe  the  preparation  time  lines  would  be  considerably  shorter 
that  they  were  the  first  time. 

The  development  of  a  distributed  T&E  architecture  is  not  a  “plug-and-play”  exercise.  In  the  near 
term,  the  elements  available  for  linking  in  a  given  architecture,  were  almost  certainly  not  designed 
to  be  linked.  That  means  that  the  burden  for  making  linked  architectures  work  falls  upon  the 
interfacing  and  integrating  activities.  The  NIUs,  the  translation  software,  the  geographical 
transforms,  etc.  are  the  interface  components  which  allow  distributed  systems  to  function  with 
existing  players  today. 
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1.0  Introduction 


1.1  JADS  Overview 

The  Joint  Advanced  Distributed  Simulation  Joint  Test  and  Evaluation  (JADS  JT&E)  was 
chartered  by  the  Deputy  Director,  Test,  Systems  Engineering  and  Evaluation  (Test  and 
Evaluation),  Office  of  the  Under  Secretary  of  Defense  (Acquisition  and  Technology)  in  October 
1994  to  investigate  the  utility  of  Advanced  Distributed  Simulation  (ADS)  technologies  for 
support  of  Developmental  Test  and  Evaluation  (DT&E)  and  Operational  Test  and  Evaluation 
(OT&E).  The  program  is  Air  Force  led,  with  Army  and  Navy  participation.  The  Joint  Test  Force 
(JTF)  manning  includes  23  Air  Force,  13  Army,  and  2  Navy.  Science  Applications  International 
Corporation  and  Georgia  Tech  Research  Institute  provide  contracted  technical  support.  The 
program  is  nominally  scheduled  for  five  years. 

The  JADS  JT&E  is  directly  investigating  ADS  applications  in  three  slices  of  the  T&E  spectrum:  a 
System  Integration  Test  (SIT)  which  explores  ADS  support  of  air-to-air  missile  testing,  an  End- 
To-End  (ETE)  test  which  explores  ADS  support  for  Command,  Control,  Communications, 
Computers,  and  Intelligence  (C4I)  testing,  and  an  Electronic  Warfare  (EW)  test  which  explores 
ADS  support  for  EW  testing.  The  JTF  is  also  chartered  to  observe,  or  participate  at  a  modest 
level,  in  ADS  activities  sponsored  and  conducted  by  other  agencies  in  an  effort  to  broaden 
conclusions  developed  in  the  three  dedicated  test  areas. 

The  SIT  is  the  subject  of  this  report  and  is  described  in  the  next  section;  the  following  is  a  brief 
synopsis  of  the  ETE  and  EW  tests. 

The  ETE  will  evaluate  the  utility  of  ADS  to  complement  the  testing  of  a  C4I  system.  ADS  will  be 
used  to  provide  a  more  robust  test  environment  that  provides  more  representative  numbers  of 
threats  plus  the  complementary  suite  of  other  C4I  and  weapon  systems  with  which  die  system 
under  test  would  interact.  The  ETE  architecture  uses  both  the  Joint  STARS  aircraft  (E-8C)  and 
ground  station  module;  JANUS,  a  constructive  model  which  generates  thousands  of  virtual 
targets;  and  Army  command  and  control  (C2)  elements.  The  C2  elements  pass  aircraft 
information  to  a  virtual  fire  direction  center,  which  in  turn,  directs  the  fire  of  a  virtual  Advanced 
Tactical  Missile  System  battery  against  selected  targets.  The  ETE  test  will  execute  in  four  phases. 
Phase  I  of  the  ETE  involves  testing  and  verification  and  validation  (V &V)  of  the  simulations  in 
stand  alone  configurations.  Phase  II  moves  from  individual  aircraft  components  to  an  integrated 
laboratory  architecture.  Phase  III  incorporates  an  E-8C  aircraft  on  the  ramp  while  Phase  IV 
incorporates  a  flying  E-8C. 

The  EW  test,  meanwhile,  will  evaluate  the  utility  of  ADS  in  a  distributed  EW  test  environment. 
The  first  phase  is  open  air  testing  to  develop  a  performance  baseline  for  two  subsequent  test 
phases.  The  first  distributed  test  phase  employs  a  linked  architecture  utilizing  DoD’s  High  Level 
Architecture  (HLA)  which  includes  a  digital  simulation  model  of  the  ALQ-131  self  protection 
jammer,  threat  simulation  facilities,  and  constructive  models  which  support  replication  of  the  open 
air  environment.  In  the  second  phase,  an  installed  systems  test  facility  is  substituted  for  the  digital 
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model.  In  both  distributed  test  architectures,  system  performance  data  will  be  compared  with  live 
fly  data  for  V&V. 

1.2  Test  Overview 

The  SIT  will  evaluate  the  utility  of  using  ADS  to  support  cost-effective  testing  of  an  integrated 
missile  weapon/launch  aircraft  system  in  an  operationally  realistic  scenario.  The  purpose  of  the 
SIT  also  includes  the  evaluation  of  the  capability  of  the  JADS  Test  Control  and  Analysis  Center 
(TCAC)  to  control  a  distributed  test  of  this  type  and  to  remotely  monitor  and  analyze  test  results. 

The  SIT  consists  of  two  phases,  each  of  which  culminates  in  three  flight  missions.  The  missions 
simulate  a  single  shooter  aircraft  launching  an  air-to-air  missile  against  a  single  target  aircraft.  In 
the  Linked  Simulators  Phase  (LSP),  the  shooter,  target,  and  missile  are  all  represented  by 
simulators.  In  the  Live  Fly  Phase  (LFP),  the  shooter  and  target  are  represented  by  live  aircraft 
and  the  missile  by  a  simulator.  This  report  addresses  the  LSP  results. 

Missile  systems  selected  for  the  SIT  are  the  AIM-9  Sidewinder  for  the  LSP  and  the  AIM-120 
Advanced  Medium  Range  Air-to-Air  Missile  (AMRAAM)  for  the  LFP.  The  intent  is  to  extend 
the  SIT  results  as  being  representative  of  the  broad  class  of  precision  guided  munitions  systems. 
Future  missile  programs  which  might  benefit  include  the  AIM-9X,  Joint  Direct  Attack  Munition 
(JDAM),  and  Evolved  Sea  Sparrow  Missile  (ESSM). 
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2.0  LSP  Test  Plan  Overview 


2.1  LSP  Purpose 

The  LSP  applies  ADS  to  an  air-to-air  missile  test  program  and  involves  the  simulation  of  an 
aircraft  launching  a  missile  against  a  maneuvering  target  aircraft.  ADS  techniques  will  be  used  to 
link  manned  flight  laboratories  representing  the  aircraft  to  an  air-to-air  missile  hardware-in-the- 
loop  (HWIL)  laboratory  representing  the  missile.  This  allows  the  reaction  of  the  target  pilot  and 
aircraft  countermeasures  systems  to  the  missile  to  be  evaluated  without  endangering  the  pilot,  a 
key  potential  benefit  of  ADS  to  T&E. 

The  LSP  configuration  has  both  DT  and  OT  characteristics.  There  is  a  DT  flavor  because  an 
HWIL  facility  is  used  to  simulate  the  missile.  This  allows  the  detailed  performance  of  missile 
subsystems  to  be  monitored,  typical  of  a  DT  test.  The  OT  characteristics  of  the  LSP  result  from 
the  use  of  aircraft  simulators  performing  operationally  realistic  engagements. 

2.2  LSP  Approach 

In  the  LSP,  both  launch  and  target  aircraft  were  represented  by  manned  flight  laboratories.  These 
laboratories  were  linked  to  each  other  and  to  an  AIM-9M-8/9  HWIL  simulation  at  the  Simulation 
Laboratory  (SIMLAB)  at  China  Lake.  The  launch  aircraft  laboratory  “fired”  the  AIM-9  in  the 
SIMLAB  at  the  simulated  target  aircraft.  The  target  received  an  indication  of  the  missile’s 
presence  from  a  missile  warning  simulation  and  could  respond  by  “dropping”  flares  and 
maneuvering. 

The  F/A-18  Weapon  System  Support  Facility  (WSSF)  at  China  Lake  and  the  F-14D  Weapon 
System  Integration  Center  (WSIC)  at  Point  Mugu  were  the  shooter  and  target,  respectively. 
Missile  warning  system  functions  were  simulated  by  a  digital  model  at  the  WSIC.  Since  the  AIM- 
9  is  an  IR-seeking  missile,  flares  were  used  as  a  countermeasure.  The  IR  signatures  and  relative 
motions  of  the  target  aircraft  and  the  flares  were  simulated  by  IR  sources  in  the  SIMLAB.  Real¬ 
time  links  between  the  simulations  allowed  the  players  to  respond  to  each  other. 

The  simulation  runs  were  controlled  by  either  the  Battle  Management  Interoperability  Center 
(BMIC)  at  Point  Mugu  or  the  Test  Control  and  Analysis  Center  (TCAC)  in  Albuquerque.  The 
Control  Center  (i.e.,  either  the  BMIC  or  the  TCAC)  ensured  that  all  nodes  were  ready  for  each 
run  and  issued  the  commands  to  start  and  stop  the  runs.  Entity  state  (ES)  PDUs  from  each 
simulation  were  processed  at  the  Control  Center  to  provide  JADS  test  controllers  and  analysts 
with  real-time  stealth  node  viewing  of  the  simulated  engagement. 

Figure  2.2-1  shows  the  organizational  structure  for  reporting  and  coordination  during  the  LSP. 
Roles  and  responsibilities  of  the  organizations  in  the  figure  are  as  follows: 

-  DDT&E:  The  Deputy  Director,  Test,  Systems  Engineering  and  Evaluation  (Test  and 
Evaluation),  Office  of  the  Under  Secretary  of  Defense  (Acquisition  and  Technology) 
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oversees  the  execution  of  the  JADS  JT&E,  approves  JADS  financial  requirements,  approves 
the  Program  Test  Plan  (PTP),  and  oversees  the  analysis  and  reporting  of  test  results. 

JADS  JTD:  The  JADS  Joint  Test  Director  (JTD)  oversees  the  preparation  of  the  PTP, 
determines  the  resources  necessary  to  complete  the  JT&E,  establishes  liaisons  with  related 
programs,  conducts  the  JT&E,  and  reports  test  results  to  DDT&E.  Coordinates  JADS 
JT&E  funding  requirements  with  the  Deputy  Director,  Test  Facilities  and  Resources  (TFR), 
Office  of  the  Under  Secretary  of  Defense  (Acquisition  and  Technology). 

JADS  SIT  Team:  Responsible  to  the  JADS  JTD  for  planning  and  conducting  the  LSP. 

NAWCWPNS:  Provides  project  management  of  the  LSP  activities  at  Pt.  Mugu  and  China 
Lake,  under  the  direction  of  the  JADS  SIT  Team.  Coordinates  requests  for  AIM-9  test 
results  and  program  information  with  the  AIM-9  Joint  System  Program  Office  (JSPO). 


-  Command/Reporting 

■  ■  ■  Coordination 

Figure  2.2-1.  Linked  Simulators  Phase  Organizational  Structure 
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2.3  LSP  Objectives 


The  overall  objective  of  the  LSP  was  to  evaluate  the  utility  of  using  ADS  to  support  cost-effective 
testing  of  an  integrated  missile  weapon/launch  aircraft  system  in  an  operationally  realistic 
scenario.  This  top-level  objective  was  accomplished  through  the  following  test  objectives. 

2.3.1  Test  Objective  1:  Assess  the  validity  of  AIM-9  data  obtained  in  the  LSP  ADS 
configuration 

Under  this  objective,  the  SIT  LSP  assessed  the  validity  of  AIM-9M-8/9  data  generated  in  the  LSP 
ADS  configuration.  This  was  measured  by  determining  whether  or  not  the  LSP  was  capable  of 
providing  valid  AIM-9  data.  This  assessment  relied  on  evaluation  of  the  data  by  analysts  from  the 
AIM-9  test  program  working  together  with  the  JADS  analysts  to  execute  the  validation 
procedures  for  the  SIT  LFP. 

2.3.2  Test  Objective  2:  Assess  utility  of  LSP  ADS  configuration  for  parametric  studies 

This  objective  evaluated  a  potential  benefit  of  the  LSP  ADS  configuration  to  AIM-9  testing:  the 
ability  to  conduct  parametric  studies.  The  assessment  addressed  two  questions:  (1)  can  valid 
parametric  studies  involving  countermeasures  (CM)  effectiveness  be  conducted  with  this  ADS 
configuration?  (2)  if  so,  is  there  utility  in  using  this  ADS  configuration  for  such  studies?  The  first 
question  was  addressed  by  evaluating  the  ability  to  repeat  a  given  scenario  with  either  no  changes 
or  with  a  single  parameter  varying.  The  second  question  was  addressed  by  evaluating  the  cost 
and  efficiency  of  executing  the  parametric  studies  using  the  LSP  ADS  configuration. 

2.3.3  Test  Objective  3:  Assess  effect  of  latency  on  validity  of  test  results 

This  objective  was  to  evaluate  the  effects  of  latency  on  test  results.  An  issue  here  was  that  when 
there  was  significant  latency,  the  different  simulation  nodes  experienced  a  “differenf  ’  engagement. 
Although  the  AIM-9M-8/9  may  continue  to  successfully  intercept  the  target  in  the  engagement 
presented  to  it  at  its  node,  this  was  not  exactly  the  same  engagement  as  that  experienced  by  either 
the  target  or  the  shooter.  In  other  words,  with  “too  much”  latency,  the  planned  engagement 
cannot  be  executed.  Further,  “too  much”  latency  will  invalidate  the  results  of  a  reactive  scenario 
(missile  and  target  are  reacting  to  each  other). 

2.3.4  Test  Objective  4:  Assess  ability  of  LSP  ADS  configuration  to  support  AIM-9  testing 

This  test  objective  was  broken  into  subobjectives  as  follows. 

2.3.4.1  Test  Subobjective  4-1:  Assess  capability  of  ADS  network  to  provide  bandwidth  and 
connectivity  required  for  LSP  tests 

This  subobjective  assessed  the  ability  of  the  LSP  ADS  network  to  support  AIM-9  testing  when  it 
was  operating. 
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23.4.2  Test  Subobjective  4-2:  Assess  the  effects  of  ADS-induced  errors  on  LSP  test  results 
validity 

The  ADS-induced  errors  were  considered  included  PDUs  not  received  at  the  appropriate  node  or 
received  out  of  order,  PDUs  corrupted  during  transmission,  and  TSPI  data  errors  introduced 
during  the  coordinate  transformations  required  for  entity  state  PDUs. 

2.3.4.3  Test  Subobjective  4-3:  Assess  adequacy  of  standard  data  protocols  for  LSP  test 

The  SIT  LSP  utilized  standard  Distributed  Interactive  Simulation  (DIS)  PDUs,  where  feasible,  for 
transferring  information  between  simulation  nodes.  This  was  desirable  in  order  to  maintain  an 
open  architecture  in  which  other  DIS-compliant  live  test  ranges  or  simulation  facilities  could 
replace  those  used  in  the  LSP  in  future  ADS  tests  of  this  type.  The  adequacy  of  the  standard  DIS 
PDUs  to  provide  the  required  data  accuracy,  synchronization,  and  minimal  latency  was  assessed. 

2.3.4.4  Test  Subobjective  4-4:  Assess  reliability,  availability,  and  maintainability  of  ADS 
network 

This  subobjective  assessed  the  degree  to  which  the  LSP  ADS  network  was  available  to  support 
AIM-9  testing  and  could  be  maintained. 

2.3.4.5  Test  Subobjective  4-5:  Assess  capability  for  centralized  test  control  and  monitoring 

This  subobjective  assessed  the  capability  of  the  NAWCWPNS  BMIC  and/or  the  JADS  TCAC  to 
control  a  distributed  test  of  this  type  and  to  remotely  monitor  and  analyze  test  results. 

2.4  LSP  Methodology 

2.4.1  Scenarios 

A  baseline  scenario  was  selected  from  the  AIM-9M-8/9  Joint  Initial  Operational  Test  and 
Evaluation  (JIOT&E)  test  series  (conducted  17  May  1993  to  29  October  1993).  This  baseline 
was  profile  LPN-15  and  is  illustrated  in  Figure  2.4. 1-1.  In  this  test,  the  target  began  ejecting 
flares  as  a  CM  at  a  preset  time  relative  to  missile  launch,  rather  than  on  a  missile  warning  cue. 
The  target  began  a  3.6  g  maneuver  prior  to  launch  and  continued  this  constant  rate  turn 
throughout  the  engagement.  The  shooter  launched  the  AIM-9  missile  when  the  proper  target 
aspect  was  obtained. 

This  single  engagement  geometry  was  the  basis  for  all  trials  in  the  LSP.  The  selection  of  this 
baseline  from  the  16  live  fire  profiles  of  the  AIM-9M-8/9  JIOT&E  was  based  on  three  factors: 
(1)  die  shooter  was  an  F/A-18,  (2)  flares  were  deployed,  and  (3)  sufficient  live  fire  data  were 
available  for  V&V  of  the  LSP  trials.  Additional  details  on  LPN-15  are  in  Appendix  A. 
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•  10,400  ft /  0.72  mach 

•  58  0  angle  off  tail 

•  3.6  g  level  turn 

•  flare  countermeasures 


F/A-18C 

11,300  ft  /  0.71  mach 
0  °  angle  off  boresight 


Figure  2.4.1-1.  AIM-9M-8/9  Live  Fire  Profile  (LPN-15,  9  June  93) 

2.4.2  Planned  Test  Events 

The  LSP  test  objectives  were  to  be  accomplished  in  three  blocks  of  simulation  “mission  time.” 
These  missions  are  described  below. 

2.4.2.1  Mission  #1:  V&V  Mission 

This  mission  was  designed  to  accomplish  LSP  test  objectives  1  and  4  and  utilized  the  AIM-9M- 
8/9  JIOT&E  live  fire  test  selected  as  the  baseline,  LPN-15,  depicted  in  Figure  2.4. 1-1 .  The  pilots 
in  the  WSSF  and  WSIC  flight  simulators  attempted  to  replicate  the  five  fire  profile  by  using 
scripted  passes  and  with  two  variations:  Profile  VI  (without  flare  countermeasures)  and  Profile 
V2  (with  flare  countermeasures).  Both  profiles  were  baselined  on  the  actual  shooter  and  target 
parameters  and  event  timing  in  the  LPN-15  scenario.  Profile  VI  was  the  verification  profile 
(without  flares  because  not  using  the  SIMLAB  flare  simulator  speeds  up  the  verification  passes 
and  the  ADS  configuration  was  the  same  as  for  the  validation  profile)  and  Profile  V2  was  the 
validation  profile  for  the  V&V  Mission.  The  V&V  approach  to  be  used  involved  comparing  the 
performance  of  the  simulated  missile  (especially  its  flyout)  to  that  of  the  real  missile  to  establish 
the  validity  of  the  ADS  configuration. 

Table  2.4.2.1-1  summarizes  the  planned  V&V  Mission  test  matrix. 
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Table  2.4.2.1-1.  LSP  V&V  Mission  Planned  Test  Matrix 


Simulation  Control  Mode 

Profile/ 

Auto 

Manual 

SIMLAB 

Linked  to 

Remarks 

Repetitions 

Pre- 

Program 

Monte 

Carlo 

Replay 

No 

Flares 

WWSB 

Stand- 

Alone 

WSSF/ 

WSIC 

RUNS  TO  BE  EXECUTED  BEFORE  MISSION  #1  1 

Vl-APS/ 

10 

X 

X 

Verification:  Auto  replicate 
live  fire  profile  without  flares 

V2-APS/ 

20 

X 

X 

iBiiH 

V2-AMS/ 

25 

D 

X 

Characterize  output  variations 
due  to  initial  conditions 

RUNS  TO  BE  EXECUTED  DURING  MISSION  #1  -  DAY  1  _ 1 

Vl-MNL/ 

10 

D 

X 

Verification:  Manual  replicate 
live  fire  profile  without  flares 

V2-MML/ 

30 

X 

X 

Rl 

INS  TO  BE  EXECUTED  DURING  MISSION  #1  -  DAY  2  1 

Vl-MNL/ 

10 

X 

X 

Verification:  Manual  replicate 
live  fire  profile  without  flares 

V2-ARL/ 

20 

a 

X 

Validation:  Auto  replay  “best” 
manual  replication  trial 

RUNS  TO  BE  EXECUTED  AFTER  MISSION  #1 

V2-ARS/ 

10 

| 

X 

X 

Characterize  variations  due  to 
SIMLAB  only 

Features  of  this  test  matrix  were  as  follows. 

-  The  two  profiles  (VI  and  V2)  were  to  be  exercised  in  both  a  “manual”  mode  and  an 
“automatic”  mode,  as  follows: 

—  Manual  Mode.  Each  simulator  was  initialized  with  a  pre-established  set  of  initial 
conditions  that  have  been  derived  from  data  produced  in  the  actual  LPN-15  flight  test. 
The  shooter  and  target  simulators  are  each  flown  and  controlled  by  pilots.  Each 
simulator  was  initially  in  a  “hold”  mode  (not  running  dynamically)  and  loaded  with  its 
respective  set  of  initial  conditions  representative  of  a  flight  test  being  conducted  over 
Holloman  AFB  (world  coordinates  and  terrain  data  base).  Dynamic  running  of  each 
simulator  was  started  manually  by  local  operators  when  its  respective  “start”  command 
is  received  from  the  BMIC/TCAC.  Flares  were  manually  released. 

-  Automatic  Mode  (Simulators  Linked)  .  Each  simulator  was  to  be  locally  driven  by  a 
playback  of  the  specific  recorded  subset  of  data  that  was  pertinent  to  that  simulator. 
That  is,  the  WSSF  was  to  be  driven  by  shooter  data,  and  the  WSIC  was  to  be  driven 
by  target  data.  In  this  mode  there  were  no  pilots  flying  or  controlling  the  shooter  and 
target  simulators  (except  for  manual  trigger  squeeze  by  the  shooter  and  manual  flare 
release  by  the  target).  The  SIMLAB  (missile)  was  driven  in  each  run  by  inputs  from 
the  shooter  and  target.  Control  of  shooter  and  target  trajectories  and  timed  events 
(except  for  trigger  squeeze  and  flare  release)  were  all  embedded  in  the  playback  data. 
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-  The  test  matrix  was  to  be  executed  over  a  number  of  days,  including  runs  before  and  after 
the  Mission  #1  test  days. 

SIMLAB  standalone  runs  were  to  be  done  before  the  Mission  #1  test  days  (Profiles  VI- 
APS  and  V2-APS).  These  runs  were  to  be  automatic,  but  without  linking  the  SIMLAB  to 
the  other  simulators.  They  were  to  be  executed  by  either  preprogramming  the  SIMLAB 
with  the  LPN-15  profile,  including  all  timed  events,  or  by  Monte  Carlo  sampling  of  the 
WSSF  and  WSIC  inputs  about  the  LPN-15  profile.  These  runs  were  to  be  used  to  (1) 
verify  SIMLAB  operation,  (2)  provide  a  baseline  for  missile  flyout  results  from  the  linked 
configuration,  and  (3)  develop  criteria  for  initial  (i.e.,  launch)  conditions  and  for  the 
validity  of  Mission  #1  missile  flyout  results. 

-  Mission  #1  was  to  be  executed  over  two  days. 

—  The  first  day  of  Mission  #1  was  to  begin  with  the  verification  profile  (VI)  to  ensure 
the  linked  configuration  was  operating  properly.  Next,  the  validation  profile  (V2)  was 
to  be  manually  replicated  a  number  of  times  (using  scripted  passes)  to  obtain  a  single 
trial  which  best  replicates  the  live  fire  profile,  LPN-15. 

-  The  second  day  of  Mission  #1  again  was  to  begin  with  the  verification  profile.  Then, 
the  best  trial  from  the  previous  day  was  to  be  automatically  replayed  a  number  of  times 
to  determine  the  run-to-run  variations  in  the  SIMLAB  output  when  the  shooter  and 
target  simulator  inputs  were  held  constant  and  when  the  ADS  network  was  used. 

SIMLAB  standalone  runs  were  to  be  done  after  the  Mission  #1  test  days  (Profile  V2- 
ARS).  These  runs  were  to  be  executed  by  replaying  the  same  manual  run  which  best 
replicated  LPN-15  to  determine  the  run-to-run  variations  in  the  SIMLAB  output  when  the 
shooter  and  target  simulator  inputs  were  held  constant  (for  “best”  manual  run  conditions) 
and  when  the  ADS  network  was  not  used. 

2.4.2.2  Mission  #2:  Parametric  Study  Mission 

This  mission  was  designed  to  accomplish  LSP  test  objectives  2  and  4  and  was  to  utilize  variations 
in  the  five  fire  test  baseline.  A  key  feature  in  parametric  studies  was  the  replay  of  a  given  scenario 
with  either  no  changes  or  with  a  single  parameter  varying. 

The  basic  scenario  to  be  simulated  in  this  mission  is  shown  in  Figure  2.4.2.2-1.  The  engagement 
was  to  begin  with  the  same  initial  conditions  as  in  the  baseline,  LPN-15  (Fig.  2.4.1-1).  The 
difference  was  to  be  that  the  target  now  received  a  missile  warning  detection  cue  (either  at  or 
after  launch)  and  employed  CM  based  on  that  cue  (rather  than  at  a  preset  time,  as  in  LPN-15). 
The  CM  was  to  always  involve  ejection  of  flares,  but  could  also  include  evasive  maneuvers. 
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e  Countermeasure 


Figure  2.4.2.2-1.  Scenario  Simulated  in  Parametric  Study  Missions 

Eight  flight  profiles  were  planned  for  the  Parametric  Mission.  Profile  VI  (without  flare 
countermeasures)  and  Profile  V2  (with  flare  countermeasures)  used  for  the  V&V  Mission  tests 
were  to  also  be  the  baseline  cases  for  the  Parametric  Mission.  In  this  mission  Profiles  VI  and  V2 
were  to  be  run  exactly  as  they  were  in  the  V&V  Mission  except  they  were  to  be  designated  as 
Profiles  PI  and  P2,  respectively.  Parametric  variations  on  the  target  “g”  and  on  the  times  of  the 
target  maneuver,  the  missile  warning  cue,  and  the  countermeasures  (flare)  release  were  then  to  be 
run  as  Profiles  P3  through  P8.  The  test  matrix  for  this  mission  is  given  in  Table  2.4.2.2-1 . 


Table  2.4.2.2-1.  Parametric  Study  Mission  Test  Matrix 


Profile/ 

Repetitions 

VARIABLE  PARAMETER 

REMARKS 

Flare  Dispense  Time 

Target  Maneuver  Time 

N/C 

+0  s 

+2  s 

+4  s 

N/C 

+0  s 

+2  s 

+4  s 

Pl/10 

■ 

■ 

X 

■ 

■ 

■ 

Baseline  verification  profile  (VI) 

No  flares 

P2/10 

X 

■ 

X 

■ 

Baseline  validation  profile  (V2) 

No  warning  cue 

P3/10 

X 

■ 

■ 

X 

■ 

■ 

■ 

Automatic  flare  dispense  * 

Warning  cue  at  +0  sec 

P4/10 

X 

X 

■ 

■ 

Automatic  flare  dispense 

Warning  cue  at  +2  sec 

P5/10 

■ 

■ 

X 

X 

■ 

■ 

Automatic  flare  dispense 

Warning  cue  at  +4  sec 

P6/10 

X 

■ 

■ 

■ 

X 

■ 

Automatic  flare  dispense 

Maneuver:  increase  turn  to  7+  g 

Warning  cue  at  +0  sec 

P7/10 

■ 

X 

■ 

■ 

■ 

X 

■ 

Automatic  flare  dispense 

Maneuver:  increase  turn  to  7+  g 

Warning  cue  at  +2  sec 

P8/10 

■ 

■ 

X 

■ 

■ 

■ 

X 

Automatic  flare  dispense 

Maneuver:  increase  turn  to  7+  g 

Warning  cue  at  +4  sec 

Note:  N/C  =  no  change  from  baseline  (V&V)  profile 


Features  of  this  mission  were  to  be  as  follows. 


This  mission  was  to  utilize  the  same  initial  engagement  geometry  as  the  V&V  Mission. 
The  V&V  profile  was  to  be  modified  by  changing  the  flare  release  time  and  by  executing 
an  evasive  maneuver  (in  some  profiles)  after  the  missile  was  launched. 

-  The  mission  was  to  begin  with  repetitions  of  the  verification  and  the  validation  passes 
from  the  V&V  Mission  to  ensure  the  test  configuration  was  working  properly.  Note  that 
the  V&V  Mission  did  not  utilize  warning  cues  to  dispense  the  flares  (they  were  dispensed 
at  a  preset  time  during  the  engagement). 

-  Half  of  the  runs  (i.e.,  5  runs)  for  each  profile  were  to  be  manually  flown  by  the  simulator 
pilots  on  the  first  day  of  Mission  #2.  The  manual  trial  for  each  profile  which  best  matched 
the  planned  profile  was  to  be  automatically  replayed  on  the  second  day  of  Mission  #2  for 
the  remaining  runs. 

-  The  countermeasures  (flare  release  and  target  maneuver)  were  to  be  executed  when  the 
warning  cue  was  received  by  the  target. 

-  Automatic  flare  release  was  to  be  initiated  by  automatic  transmission  of  a  Fire  (Flare) 
PDU  from  the  WSIC. 

-  In  all  passes,  the  target  aircraft  was  already  in  a  3.6  g  turn  at  missile  launch  (see  Fig.  2.4.1- 
1).  In  the  runs  involving  the  evasive  maneuver,  the  target  pilot  was  to  increase  the  turn 
rate  by  at  least  3.4  g  to  7  g,  or  more,  upon  receipt  of  the  warning  cue. 

The  evaluation  criteria  here  were  to  be  to  determine  if  the  engagement  can  be  repeatedly 
initialized  within  acceptable  tolerances,  if  the  parameters  can  be  varied  in  a  controlled  manner,  and 
if  the  aircraft  profiles  can  be  manually  replicated  to  the  degree  necessary  to  produce  repeatable 
results  when  parameters  were  not  varied. 

2.4.2.3  Mission  #3:  Latency  Study  Mission 

An  assumption  for  the  Latency  Mission  was  that  the  V&V  Mission  was  successfully  completed 
and  showed  that  the  latency  of  the  baseline  configuration  was  small  enough  to  give  valid  results. 
A  reactive  scenario  was  to  be  utilized  in  which  the  target  reacted  to  the  missile  from  cues 
generated  by  the  missile  warning  system  model.  Note  that  this  would  not  be  the  V&V  scenario 
(which  did  not  involve  the  missile  warning  system  model).  Rather,  a  scenario  was  to  be  selected 
from  the  Parametric  Mission  in  which  the  warning  system/CM  clearly  affected  die  missile 
performance. 

One  flight  profile  was  planned  for  the  Latency  Study  Mission:  the  reactive  profile  selected  from 
the  Parametric  Study  Mission,  designated  as  Profile  LI  for  the  Latency  Mission  tests.  The  latency 
study  was  to  be  performed  by  first  replicating  this  profile  for  the  baseline  minimum  latency  case 
(inherent  system/network  latency  only)  and  noting  the  engagement  results  for  each  node.  Next 
the  profile  was  to  be  repeated  with  an  adjustable  time  delay  at  the  WSIC  node  only  and  again  the 
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engagement  results  for  each  node  were  to  be  noted.  The  delay  was  to  be  between  the  network 
and  the  WSIC  network  interface  unit  (NIU)  and  was  to  affect  both  incoming  and  outgoing  PDUs. 
The  profile  was  to  be  repeated  with  additional  increments  of  time  delay  at  the  WSIC  node.  The 
test  matrix  for  this  mission  is  given  in  Table  2.4.2. 3-1. 

Table  2.4.2.3-1.  Latency  Study  Mission  Test  Matrix 


Features  of  this  mission  were  to  be  as  follows. 

-  The  mission  was  to  begin  with  repetitions  of  the  selected  profile  from  the  Parametric 
Mission  to  ensure  the  test  configuration  was  working  properly.  The  profile  was  first  to  be 
automatically  replayed  in  the  stand-alone  SIMLAB  configuration  (profile  Ll-S  in  Table 
2.4.2. 3-1).  Next  the  profile  was  to  be  manuallyreflown  by  the  WSSF  and  WSIC  pilots. 

-  Profile  Ll-S  also  was  to  serve  as  a  baseline  of  missile  performance  data  that  was  free  of 
variations  in  initial  conditions  and  free  of  any  network  latencies.  (Note:  This  test  was 
necessary  to  establish  a  “truth”  baseline  for  missile  performance  because  corresponding 
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missile  data  from  the  “baseline”  profile  from  the  Parametric  Mission  would  inherently 
contain  the  effects  of  “minimum”  latency  across  the  network.) 

-  The  time  delays  were  to  be  introduced  only  at  the  WSIC  node.  The  link  between  the 
WSSF  and  the  SIMLAB  was  not  to  be  affected.  The  reason  for  not  adding  time  delays 
between  the  WSSF  and  SIMLAB  were  that  there  were  two  links  between  these  facilities 
(one  for  PDU  data  and  one  for  the  MIL-STD-1553  Stores  Management  System  (SMS) 
data),  delays  would  have  to  added  to  both  links  simultaneously,  and  significant  delays 
could  not  be  added  to  the  SMS  link  without  affecting  the  proper  operation  of  the  SMS 
system.  Adding  delays  only  at  the  WSIC  node  would  have  simulated  the  situation  of 
moving  the  target  node  to  a  geographically  remote  location,  while  keeping  the  shooter  and 
missile  co-located. 

The  values  of  time  delay  were  to  be  set  in  a  computer  in  the  WSIC  and  which  was  located 
between  the  Cisco  router  and  the  rest  of  the  WSIC  network.  This  was  to  be  the  first 
processing  point  in  the  WSIC  encountered  by  incoming  PDUs  and  the  last  processing 
point  encountered  by  outgoing  PDUs.  The  computer  was  to  buffer  the  PDUs  until  the 
preset  delay  time  had  elapsed.  At  that  time,  the  computer  was  to  transmit  the  PDUs.  The 
actual  simulation-computer-to-simulation-computer  latency  was  to  be  measured  before 
each  run  by  generating  a  fire  flare  signal  in  the  WSIC  (with  generation  time  stamp)  and 
recording  its  input  into  another  simulation  computer  (with  input  time  stamp).  Latency 
between  the  WSIC  and  the  BMIC/TCAC  was  to  be  determined  from  the  time  stamp  when 
the  Fire  (Flare)  PDU  was  recorded  in  the  PDU  logger  located  in  the  BMIC/TCAC. 

Increments  of  time  delay  to  be  implemented  in  Test  1  (first  day  of  testing)  were  to  be 
determined  after  analysis  of  the  V&V  and  Parametric  Mission  results.  These  were  to  be 
coarse  time  delay  steps  which  should  have  ensured  that  significant  latency  effects  were 
noted  in  the  results.  Preliminary  analysis  indicated  that  time  delay  steps  between  10  msec 
and  100  msec  should  result  in  the  nodes  disagreeing  on  the  missile  hitting  or  missing  the 
target. 

Increments  of  time  delay  to  be  implemented  in  Test  2  (second  day  of  testing)  were  to  be 
determined  after  quick-look  analysis  of  the  Test  1  results.  These  were  to  be  fine  time 
delay  steps  selected  to  better  identify  the  onset  of  significant  latency  effects. 

As  the  time  delay  was  increased,  the  engagement  viewed  at  each  on  the  four  nodes  (WSSF, 
SIMLAB,  WSIC,  BMIC/TCAC)  was  expected  to  increasing  differ  in  terms  of  launch  range,  flare 
release/target  maneuver  time,  and  miss  distance.  Latency  was  to  have  clearly  affected  the 
engagement  outcome  if  one  node  perceived  a  “hif  ’  (missile  passed  within  lethal  radius  of  target) 
and  another  node  perceived  a  “miss”  (missile  passed  outside  lethal  radius  of  target). 
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2.4.3  Test  Configuration 
2.4.3.1  Simulation  Sites 
WSSF 

The  launch  aircraft  was  simulated  by  the  F/A-l 8  WSSF  manned  flight  laboratory.  The  WSSF  is  a 
computer-based  laboratory  with  real  integrated  avionics  used  to  perform  system  and  subsystem 
level  software  development,  T&E  of  the  avionics  embedded  computer  systems  and  their 
associated  Operational  Flight  Software  (OFS).  The  WSSF  is  a  completely  integrated  facility 
capable  of  supporting  system  engineering,  software  engineering,  avionics  and  weapons 
integration,  software  maintenance  ,  and  incorporation  of  new  systems,  technology  ,  and  concepts. 
This  facility  is  the  primary  tool  used  to  provide  quick  response  investigations  of  Fleet  reported 
problems,  developmental  testing,  V&V,  safety  of  flight  testing,  correction  of  errors  and 
deficiencies,  investigation  of  changes,  and  integration  and  test  of  new  technology  and  weapons. 
Although  designed  to  meet  the  specific  requirements  of  the  F/A-l 8  aircraft  platform,  the  WSSF 
generally  supports  the  following  test  capability  requirements:  rapid  pre-  and  post-flight  testing  in 
support  of  open  air  range  testing,  hot  bench  mockup  accommodation,  MIL-STD-1553  avionics 
bus  simulation,  MIL-STD-1760  weapons  interface  bus  simulations,  high  fidelity  (6 -DOF)  flight 
motion  simulations,  System-Under-Test  (SUT)  data  collection,  real-time  interrelated  scenario 
parameter  measurement,  and  real-time  performance  monitoring,  control,  and  recording  for  MIL- 
STD-1553  and  -1760  based  systems.  A  variety  of  test  equipment  is  used  including  weapons 
simulators,  computer  support  equipment,  6  -DOF  airframe  and  engine  simulations,  environment 
simulations,  and  avionics  sensor  simulations  to  stimulate  HWIL  throughout  a  range  of  operational 
scenarios.  A  unique  flight  test  data  playback  capability  exists  where  data  logged  in  flight  can  be 
replayed  in  the  lab  oratory  for  regression  testing.  The  WSSF  can  also  be  interconnected  with 
remote  NAWC  WPNS  laboratories  and  open  air  ranges  to  extend  the  dynamic  test  capabilities  of 
the  facilities. 

SIMLAB 

The  SIMLAB  simulated  the  launch  and  flyout  of  the  AIM-9.  The  active  guidance  of  the  AIM-9 
will  be  simulated  by  stimulating  the  AIM-9  seeker  with  an  IR  scene  which  dynamically  responds 
to  the  target  aircraft  maneuvers.  The  SIMLAB  is  a  fully  equipped,  state-of-the-art,  HWIL 
simulation  facility  for  testing  actual  missile  hardware  in  an  environment  that  closely  simulates  a 
real-world  flight.  The  laboratory  consists  of  a  Carco  three  axis  flight  motion  simulator,  high 
speed  simulation  computers,  and  targeting  systems  for  generating  radar  and  infrared  signatures 
to  simulate  targets  and  countermeasures.  These  facilities  are  used  for  system  level  evaluation  of 
full  closed-loop  missile  performance,  including  sensor,  guidance  and  control,  signal  processing, 
and  countermeasures/counter-countermeasures  capability. 

WSIC 

The  target  aircraft  was  simulated  by  the  WSIC  F-14D  manned  flight  laboratory.  The  WSIC  is  a 
systems  level  test  facility  for  the  F-14D  weapons  control  systems.  This  facility  is  used  to  measure 
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the  functionality  of  F-14D  tactical  software,  to  support  hardware  integration  testing  ,  and  to 
support  the  system  level  software  integration  of  subsystem  and  module  software  packages 
developed  by  prime  contractors  and  government  activities.  The  F-14  WSIC  provides  a  complete 
simulation  and  testing  center  for  the  F-14;  secondary  functions  include  data  reduction  and  support 
to  additional  projects  from  the  Defense  Modeling  and  Simulation  Organization  (DMSO)  and  the 
Advanced  Research  Projects  Agency  (ARPA).  The  center  provides  test  beds  to  develop  new  F- 
14  tactical  software,  integrate  new  weapons  systems  into  the  F-14  aircraft  ,  and  perform  V&V  of 
the  F-14  tactical  software.  These  functions  are  performed  in  any  of  four  developmental 
environments:  full  software  simulation,  stand-alone  (single-subsystem,  simulated  system),  partial 
integration  (groups  of  subsystems  with  the  remainder  simulated),  and  full  integration.  Each 
laboratory  uses  real-time  simulations  to  stimulate  weapon  system  inputs  in  a  controlled  laboratory 
environment .  Dynamic  real-time  mathematical  models  provide  Weapon  Replacement  Assemblies 
(WRAs)  not  used  in  the  laboratory  for  user  convenience.  Real-time  and  post-flight  data 
monitoring  and  analysis  tools  are  available  for  users  to  completely  ascertain  system  performance. 
The  WSIC  supports  off-site  RDT&E  projects  using  dedicated  T1  lines  as  well  a  Red  Gateway  on 
the  Defense  Simulations  Intemet(DSI)  to  provide  the  necessary  laboratory  internetting. 

2.43.2  LSP  Network 

The  LSP  test  configuration  is  shown  in  Figure  2.4.3. 2-1,  and  the  information  flow  during  the  LSP 
is  shown  in  Figure  2.4.3. 2-1.  Each  simulation  node  was  DIS  compliant  and  encrypted.  The 
network  data  exchange  protocol  was  DIS  Version  2.0.4,  except  for  the  Stores  Management 
System  (SMS)  data  exchange  between  the  F/A-18  WSSF  and  the  AIM  -9  SIMLAB.  This  link 
used  the  MIL-STD-1553  protocol,  because  no  suitable  DIS  protocol  exists  for  these  data  and 
because  this  exchange  was  only  between  the  WSSF  and  the  SIMLAB. 

Before  launch,  the  WSSF  and  the  SIMLAB  exchanged  SMS  data  over  a  dedicated  link.  This  link 
represented  an  extension  of  the  F/A-18  avionics  bus  consisting  of  MIL-STD-1553  data  '.  The 
SMS  computer,  LAU-7  launcher  rail,  and  the  umbilical  cable  between  it  and  the  AIM-9  were  all  in 
the  SIMLAB.  The  launch  signal  from  the  F/A-18  passed  over  the  SMS  link  to  command  the 
launch  sequence  in  the  SIMLAB.  When  the  missile  battery  squib  firing  was  detected,  a  Fire 
(Missile)  PDU  was  generated  in  the  SIMLAB. 

State  data  for  each  of  the  players  was  in  the  form  of  ES  PDUs  generated  at  each  player’s  node 
and  passed  to  all  other  nodes.  The  target  ES  PDUs  were  used  in  the  SIMLAB  to  dynamically 
control  the  representation  of  the  target  in  the  IR  scene  presented  to  the  AIM-9  seeker.  As  the 
AIM-9  underwent  its  simulated  flyout,  its  trajectory  information  (position  versus  time)  was 
converted  into  ES  PDUs  in  the  SIMLAB.  When  the  SIMLAB  stopped  sending  out  missile  ES 
PDUs,  a  Detonation  PDU  was  generated  in  the  SIMLAB. 


'  The  prelaunch  SMS  data  exchange  between  the  WSSF  and  the  SIMLAB  shall  be  as  currently 
implemented  per  the  F/A-18  WSSF  MIL-STD-1553  Networking  System  (MNS-1)  Interface 
Control  Document  (ICD)  dated  14  March  1996. 
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Figure  2.4.3.2-1.  Linked  Simulators  Phase  Test  Configuration 
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Figure  2.43.2-2.  Linked  Simulators  Phase  Network  Information  Diagram 
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The  detailed  network  hardware  diagram  for  the  LSP  is  given  in  Appendix  D,  Annex  2. 

2.4.3.3  Test  Control  and  Monitoring 

The  LSP  investigated  four  areas  of  test  control:  aircraft  control,  test  article  and  associated  system 
monitoring,  communications,  and  test  procedures.  The  thrust  of  the  test  control  investigation  was 
to  determine  how  much  of  the  control  function  should  reside  at  the  individual  nodes,  how  much 
needs  to  be  at  the  central  control  node,  and  what  communications  were  required  between  the 
nodes. 

The  LSP  used  only  one  basic  engagement  geometry.  The  initial  setup  (position,  attitude,  altitude, 
airspeed  and  heading)  for  this  geometry  was  programmed  into  each  simulator.  At  the  beginning 
of  each  run,  each  simulator  could  be  reset  to  these  initial  conditions.  During  the  manual  passes, 
the  aircraft  controller  at  the  BMIC/TCAC  used  a  two-dimensional  display  to  vector  the  aircraft 
into  positions  from  which  the  pilots  could  fly  the  final  portion  of  the  intercept  to  a  firing  position. 
This  aircraft  positions,  altitude,  and  airspeed  were  displayed,  and  “tails”  were  used  to  provide  a 
position  history. 

To  control  the  test,  two-way  communications  with  the  pilots,  the  person  monitoring  the 
equipment  at  each  site,  and  various  analysts  were  required.  For  the  LSP,  open  telephone  lines  and 
a  low  bit  rate  video  system  were  used.  The  test  controller  started  and  stopped  each  run  using 
voice  commands.  The  status  of  each  node  was  also  reported  using  the  voice  circuits. 

Test  control  procedures,  test  cards,  and  checklists  are  at  Appendix  E  of  this  document. 

2.4.3.4  Instrumentation 

The  same  instrumentation  at  a  given  site  was  used  for  all  LSP  missions  (see  Table  2.4.3. 4-1) 
Time  stamp  considerations  were  as  follows: 

-  Both  China  Lake  and  Point  Mugu  use  cesium  clocks  as  time  sources.  These  clocks  are 
corrected  periodically  to  Greenwich  Mean  Time  (GMT)  which  is  provided  by  the  Global 
Positioning  System  (GPS)  satellite  constellation.  A  coded  time  signal  is  broadcast 
throughout  each  range.  Each  facility  which  requires  accurate  time  stamps  has  an  antenna 
which  receives  this  signal.  Propagation  delays  are  accounted  for  and  the  signal  in  IRIG 
format  is  available  in  the  facility.  For  the  SIT  the  IRIG-B  signal  is  converted  to  serial  and 
provided  as  an  input  to  the  data  loggers. 

The  time  source  is  accurate  to  within  1 50  nanoseconds  of  GMT.  The  serial  signal  is  used 
by  the  Silicon  Graphics,  Inc.  (SGI)  Indy  computers  in  consonance  with  the  drift  file  to 
provide  a  time  source  accurate  to  the  order  of  100  microseconds.  Since  all  of  the  latencies 
measured  will  be  greater  than  10-20  milliseconds  it  was  decided  that  a  time  source 
accuracy  of  1  millisecond  would  be  sufficient  for  SIT  purposes.  This  also  permitted  using 
standard  IRIG,  GPS,  and  DIS  PDU  formats  for  time  stamps  since  the  least  significant  bit 
in  each  of  these  formats  provides  a  precision  of  1  millisecond. 
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Table  2.4.3.4-1.  LSP  Instrumentation  Requirements 


Location 

Description 

Function 

F/A-18  WSSF 

IRIG-B  translator/generator 

Test  coordination 

4  mm  tape  recorder 

Data  transfer 

HUD  video  camera 

Capture  HUD  displays 

HUD  video  recorder  (VCR) 

Capture  HUD  displays 

DIS  DataLogger 

Record  PDUs 

DataLogger  software 

Record  PDUs 

F-14D  WSIC 

IRIG-B  translator/generator 

Test  coordination 

4  mm  tape  recorder 

Data  transfer 

HUD  video  camera 

Capture  HUD  displays 

HUD  video  recorder  (VCR) 

Capture  HUD  displays 

DIS  DataLogger 

Record  PDUs 

DataLogger  software 

Record  PDUs 

SIMLAB 

IRIG-B  translator/generator 

Test  coordination 

4  mm  tape  recorder 

Data  transfer 

9  -  track,  1/2”  magnetic  tape  recorder 

Record  hi-fidelity 
simulation  data 

Acquire  missile  TM  data 

Real-time  chart  recorders/displays 

Test  monitor 

DIS  DataLogger 

Record  PDUs 

DataLogger  software 

Record  PDUs 

TCAC 

IRIG-B  translator/generator 

Test  coordination 

Stealth  Viewer 

Test  coordination 

Video  camera(s) 

Capture  displays 

Video  recorder(s) 

Capture  displays 

DIS  DataLogger 

Record  PDUs 

DataLogger  software 

Record  PDUs 

2.4.4  LSP  Assumptions  and  Limitations 

The  following  limitations  apply: 

-  The  SIMLAB’s  IR  target  presentation  system  presented  the  target  as  a  point  IR  source. 
This  limited  the  validity  of  terminal  engagement  results  when  the  target  should  be 
represented  by  an  extended  and  structured  IR  source. 

The  simulations  at  the  different  laboratories  could  not  be  synchronized.  This  included  lack 
of  synchronization  in  the  start  of  the  simulations  and  in  the  individual  frames  of  the 
simulations.  This  prevented  reliable  performance  of  the  automatic  replay  runs. 

-  The  manned  aircraft  laboratories  did  not  have  totally  realistic  flying/handling  qualities 
when  compared  to  the  actual  aircraft  and  did  not  have  sophisticated  visual  presentation 
systems.  This  limited  their  use  to  scripted,  rather  than  free  play,  scenarios. 


-  The  missile  HWIL  laboratory  did  not  allow  a  wide  range  of  motion  in  order  to  handle 
large  angles  off  and/or  large  angle  rates.  This  also  precluded  the  use  of  free  play 
scenarios. 

2.5  LSP  Planned  Schedule 
2.5.1  Top  Level  Schedule 

The  schedule  of  top  level  tasks  for  the  LSP  is  given  in  Figure  2.5. 1-1.  The  following  tasks  are 
indicated  in  the  schedule: 

Task  1  Pre-Test  Documentati  on:  Prepare  documentation  of  test  definition,  requirements,  and 
plans.  Joint  JADS/  NAWCWPNS  task. 

Task  2  Infrastructure  Development:  Implement  the  NAWCWPNS  Real-Time  Network 
(NRNet)  to  link  the  NAWCWPNS  facilities  for  the  LSP.  NAWCWPNS  task. 

Task  3  SIMLAB  Enhancement:  Modify  SIMLAB,  as  required.  NAWCWPNS  task. 

Task  4  WSIC  Enhancement:  Modify  WSIC,  as  required.  NAWCWPNS  task. 

Task  5  WSSF  Enhancement:  Modify  WSSF,  as  required.  NAWCWPNS  task. 

Task  6  BMIC  Preparations:  Prepare  BMIC  as  the  control  node  for  Mission  #1 .  Joint  task. 
Task  7  Integration  Testing:  Verify  links  between  facilities  and  rehearse  missions.  Joint  task. 
Task  8  TCAC  Preparations:  Prepare  TCAC  as  the  control  node  for  Missions  #2  and  #3.  JADS 
task. 

Task  9  Test  Missions:  Execute  the  three  LSP  missions.  Joint  task. 

Task  10  Post-Test  Documentation:  Document  the  results  of  the  LSP  execution.  Joint  task. 


Task  Name 

Start 

Finish 

FY96 

FY97 

Qtr  2  1  Qtr  3  1  Qtr4 

Qtr  1  1  Qtr  2 

1  Pre-Test  Documentation 

1/15/96 

8/1/96  1 

2  Infrastructure  Development 

2/5/96 

6/21/96 

3  SIMLAB  Enhancement 

2/14/96 

6/21/96 

4  WSIC  Enhancement 

2/14/96 

6/21/96 

5  WSSF  Enhancement 

2/5/96 

5/31/96 

6  BMIC  Preparations 

4/10/96 

8/1/96 

7  Integration  Testing 

6/3/96 

8/1/96 

8  TCAC  Preparations 

8/16/96 

9/27/96 

9  Test  Missions 

8/5/96 

12/3/96 

10  Post-Test  Documentation 

8/9/96 

3/31/97 

Figure  2.5.1-1.  Linked  Simulators  Phase  Activities  and  Schedule 
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2.5.2  Pre-Test  Detailed  Schedule 

The  following  top  level  tasks  in  Figure  2.5. 1-1  are  pre-test  tasks: 


Task  1 

Pre-Test  Documentation 

Task  2 

Infrastructure  Development 

Task  3 

SIMLAB  Enhancement 

Task  4 

WSIC  Enhancement 

Task  5 

WSSF  Enhancement 

Task  6 

BMIC  Preparations 

Task  7 

Integration  Testing 

Task  8 

TCAC  Preparations 

The  detailed 

schedule  of  each  of  these  follows. 

Task  1  Pre-Test  Documentation.  The  detailed  schedule  for  this  task  is  shown  in  Figure 

2.5.2-1 .  The  following  tasks  and  activities  are  indicated  in  the  schedule: 

1.1  Integration  Test  Plan:  NAWCWPNS  develop  the  ITP  which  provides  a 
description  of  data  and  functional  requirements,  modifications  of  NAWCWPNS 
test  facilities,  tasking,  an  implementation  schedule,  and  a  list  of  major  milestones 
for  the  LSP  effort. 

1.2  V&V  Plan:  NAWCWPNS  develop  the  V&V  Plan  which  documents  the  detailed 
approach  for  verifying  the  LSP  ADS  configuration  during  pre-test  buildup  and  for 
validating  the  AIM-9M-8/9  results  obtained  during  Mission  #1. 

1.3  Laboratoiy  Test  Plan:  NAWCWPNS  develop  the  LTP  whi  ch  provides  a  more 
detailed  test  plan,  including  test  method  and  conduct,  data  reduction  and  analysis, 
and  requirements  for  data  and  instrumentation. 

1.4  Test  Activity  Plan:  JADS  prepare  the  TAP  which  combines  JADS  activities  and 
objectives  with  NAWCWPNS  plans  and  activities. 


Task  Name 

Start 

Finish 

FY96 

Qtr2  Qtr  3  1  Qtr4 

1  Pre-Test  Documentation 

1/15/96 

8/1/96 

1.1  Integration  Test  Plan 

1/15/96 

4/12/96 

1.2  V&V  Plan 

2/26/96 

4/12/96 

■■ 

1.3  Laboratory  Test  Plan 

3/4/96 

5/15/96 

1 .4  Test  Activity  Plan 

3/1/96 

8/1/96 

Figure  2.5.2-1.  Pre-Test  Documentation  Activities  and  Schedule 


Task  2  Infrastructure  Development.  The  detailed  schedule  for  this  task  is  shown  in  Figure 
2. 5.2-2.  The  following  tasks  and  activities  are  indicated  in  the  schedule: 
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2.1  Implement  H/W  and  Comm  Links:  Hardware  and  communications  links  at  China 
Lake  and  at  Pt.  Mugu  shall  be  implemented  by  NAWCWPNS  using  NKNet.  The 
T1  linking  China  Lake  to  Pt.  Mugu  is  already  in  place.  Routers  and  CSU/DSU 
will  be  procured  and  installed  as  required. 

2.2  Program  Routers:  The  NRNet  routers  will  be  set  up  to  run  TCP/IP. 

2.3  Implement  Secure  Operations:  An  NRNet  Memorandum  of  Agreement  (MOA) 
shall  be  developed  by  NAWCWPNS  to  establish  agreement  among  the  facilities 
concerning  secure  operation  and  cryptographic  (COMSEC)  keying  material 
handling.  COMSEC  keying  material  will  be  issued  to  each  facility  to  support 
secure  operations. 

2.4  NRNet  Checkout:  Implementation  of  the  NRNet  for  the  LSP  will  be  verified  by 
checkout  of  the  links  between  the  facilities. 


Task  Name 

Start 

Finish 

FY96 

Qtr  2  1  Qtr  3  i  Qtr4 

2  Infrastructure  Development 

2/5/96 

6/21/96 

2.1  Implement  H/W  and  Comm  Links 

2/5/96 

4/19/96 

■■ 

2.2  Program  Routers 

2/19/96 

6/21/96 

2.3  Implement  Secure  Operations 

2/19/96 

4/19/96 

2.4  NRNet  Checkout 

5/1/96 

6/21/96 

^■i 

Figure  2.5.2-2.  Infrastructure  Development  Activities  and  Schedule 


Task  3  SIMLAB  Enhancement.  The  detailed  schedule  for  this  task  is  shown  in  Figure  2.5.2- 

3.  The  following  tasks  and  activities  are  indicated  in  the  schedule: 

3.1  Develop/Install  DIS  Node:  To  support  the  LSP,  a  DIS  node  will  be  added  to  the 
SIMLAB  IR#1  Facility.  An  HP  735  computer  system  will  be  procured  and 
installed  as  the  SIMLAB’s  DIS  node.  Modifications  will  be  made  to  the  existing 
simulation  to  support  the  addition  of  the  DIS  communication  requirement. 
Simulation  Specific  Component  (SSC)  code  will  be  developed  to  provide  the 
signal  interface  between  the  simulation  system  and  the  DIS  node’s  NIU  software, 

3.2  Calibrate  Facility:  The  operating  envelope  of  IR#1  will  be  defined  to  verify  the 
ability  of  the  simulation  to  run  the  LSP  scenarios.  The  IR  target  presentation 
system,  the  flare  generator  system,  and  the  Carco  three  axis  flight  simulator  will 
be  calibrated. 

3.3  Procure/Install  Seeker:  An  AIM-9M-8/9  seeker  will  be  procured  and  installed  in 
the  IR#1  facility. 

3.4  Develop  Data  Logging  Capability:  Simulation  software  will  be  modified,  as 
required,  to  support  data  logging  and  quick-look  data  capability. 

3.5  Internal  Testing:  The  SIMLAB  modifications  will  be  verified  during  internal 
testing  at  the  facility. 
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Task  Name 

Start 

Finish 

FY96 

Qtr2  1  Qtr  3  1  Qtr4 

3  SIMLAB  Enhancement 

2/14/96 

6/21/96 

3.1  Develop/Install  DIS  Node 

2/14/96 

5/8/96 

3.2  Calibrate  Facility 

2/14/96 

6/21/96 

3.3  Procure/Install  Seeker 

4/5/96 

5/1/96 

■ 

3.4  Develop  Data  Logging  Capability 

4/16/96 

. 5/24/96 

■■ 

3.5  Internal  Testing 

5/8/96 

5/29/96 

■ 

Figure  2.5.2-3.  SIMLAB  Enhancement  Activities  and  Schedule 


Task  4  WSIC  Enhancement.  The  detailed  schedule  for  this  task  is  shown  in  Figure  2.5.2-4. 

The  following  tasks  and  activities  are  indicated  in  the  schedule: 

4. 1  Develop/Install  DIS  Node:  To  support  the  LSP,  a  DIS  node  will  be  added  to  the 
WSIC  Facility.  Modifications  will  be  made  to  the  existing  simulation  to  support 
tire  addition  of  the  DIS  communication  requirement.  SSC  code  will  be  developed 
to  provide  the  signal  interface  between  the  simulation  system  and  the  DIS  node’s 
NIU  software. 

4.2  Design/Program  Data  Exch  Node:  The  Universal  Data  Exchange  (UDX)  in  the 
WSIC  shall  be  modified  to  accept  F-14D  1553  bus  information  required  for  the 
LSP.  The  UDX  will  also  be  modified  to  convert  the  1553  bus  data  into  units 
required  by  the  SSC  code  in  the  NIU. 

4.3  Develop  Data  Logging  Capability:  Simulation  software  will  be  modifie  d,  as 
required,  to  support  data  logging  and  recording  capability 

4.4  Develop  MWS  S/W  &  H/W:  Code  will  be  developed  to  model  functioning  of  the 
missile  warning  system.  Hardware  will  be  fabricated  to  indicate  which  quadrant 
the  missile  is  approaching  from  and  to  provide  a  warning  buzzer  to  the  WSIC 
pilot. 

4.5  Internal  Testing:  The  WSIC  modifications  will  be  verified  during  internal  testing 
at  the  facility. 


Task  Name 

Start 

Finish 

FY96 

Qtr  2  1  Qtr  3  1  Qtr4 

4  WSIC  Enhancement 

2/14/96 

6/21/96 

4.1  Develop/Install  DIS  Node 

2/14/96 

5/15/96 

4.2  Design/Program  Data  Exch  Node 

2/21/96 

5/24/96 

■■■■ 

4.3  Develop  Data  Logging  Capability 

4/16/96 

5/24/96 

4.4  Develop  MWS  H/W  &  S/W 

4/3/96 

6/21/96 

■HHH 

4.5  Internal  Testing 

5/6/96 

6/21/96 

Figure  2.5.2-4.  WSIC  Enhancement  Activities  and  Schedule 


Task  5  WSSF  Enhancement.  The  detailed  schedule  for  this  task  is  shown  in  Figure  2.5.2-5. 
The  following  tasks  and  activities  are  indicated  in  the  schedule: 
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5.1  Define  Requirements:  Requirements  for  WSSF  to  support  LSP  will  be  defined. 

5.2  Modify  DIS  Node:  The  existing  DIS  node  at  the  WSSF  will  be  modified,  as 
required,  to  support  the  LSP.  The  simulation  system  and  the  SSC  code  will  be 
modified  to  support  the  PDUs  required  for  the  LSP. 

5.3  Develop  Data  Logging  Capability:  Data  recording  specification  files  will  be 
developed  to  direct  data  recording  operations  throughout  a  test  scenario. 

5.4  Internal  Testing:  The  WSSF  modifications  will  be  verified  during  internal  testing 
at  the  facility. 


Task  Name 

Start 

Finish 

FY96 

Qtr  2  Qtr  3  Qtr4 

5  WSSF  Enhancement 

2/5/96 

5/31/96 

5.1  Define  Requirements 

2/5/96 

4/3/96 

5.2  Modify  DIS  Node 

3/6/96 

5/15/96 

5.3  Develop  Data  Logging  Capability 

5/15/96 

5/31/96 

■ 

5.4  Internal  Testing 

5/15/96 

5/31/96 

■ 

Figure  2.5.2-5.  WSSF  Enhancement  Activities  and  Schedule 


Task  6  BMIC  Preparations.  The  detailed  schedule  for  this  task  is  shown  in  Figure  2.5.2-6. 

The  following  tasks  and  activities  are  indicated  in  the  schedule: 

6.1  Define  Requirements:  Requirements  for  test  control  and  monitoring  using  the 
BMIC  will  be  defined,  including  a  test  control  concept  of  operations.  The 
concept  of  operations  will  be  consistent  with  utilization  of  the  TCAC  for  test 
control. 

6.2  Install/Configure  H/W  &  S/W:  Hardware  and  software  needed  for  test  control 
and  monitoring  will  be  installed  and/or  configured  in  the  BMIC  and  at  the 
simulation  nodes,  if  required.  The  BMIC  will  be  connected  to  the  NRNet 
configuration  linking  the  simulation  facilities. 

6.3  Develop  Control  Procedures:  Procedures  for  controlling  the  LSP  missions  w  ill  be 
developed  and  documented. 

6.4  Checkout  &  Testing:  The  BMIC  modifications  will  be  verified  during  internal 
testing  at  the  facility. 


Task  Name 

Start 

Finish 

FY96 

FY97 

Qtr3  I  Qtr  4 

Qtr  1 

6  BMIC  Preparations 

4/10/96 

8/1/96 

6.1  Define  Requirements 

4/10/96 

5/31/96 

■■ 

6.2  Install/Configure  H/W  &  S/W 

7/15/96 

7/19/96 

1 

6.3  Develop  Control  Procedures 

7/1/96 

8/1/96 

WM 

6.4  Checkout  &  Testing 

7/15/96 

8/1/96 

m 

Figure  2.5.2-6.  BMIC  Preparations  Activities  and  Schedule 
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Task  7  Integration  Testing.  The  detailed  schedule  for  this  task  is  shown  in  Figure  2.52-1. 

The  following  tasks  and  activities  are  indicated  in  the  schedule: 

7.1  Develop  Test  Procedures:  Integrated  procedures  for  executing  the  LSP  missions 
will  be  developed  and  used  during  integration  testing  and  checkout  activities. 
Each  facility  will  develop  their  own  test  procedures,  and  these  will  be  coordinated 
among  the  laboratories  using  an  iterative  process.  The  final  checkout  period  will 
be  used  to  integrate  and  consolidate  the  test  procedures. 

7.2  WSSF/SIMLAB  Exch  PDUs:  The  link  between  the  WSSF  and  SIMLAB  will  be 
verified,  along  with  the  capability  to  exchange  and  record  the  required  data. 

7.3  WSSF/WSIC  Exch  PDUs:  The  link  between  the  WSSF  and  WSIC  will  be 
verified,  along  with  the  capability  to  exchange  and  record  the  required  data. 

7.4  All  Nodes  Exch  PDUs:  The  links  between  the  WSSF,  SIMLAB,  WSIC,  and 
BMIC  will  be  verified,  along  with  the  capability  to  exchange  and  record  the 
required  data. 

7.5  Dry  Runs  w/o  Test  Control:  The  LSP  configuration  will  be  exercised  without  the 
BMIC  test  control  procedures.  Test  procedures  will  be  refined. 

7.6  Dry  Runs  w/  Test  Control:  The  ability  of  the  BMIC  to  control  the  simulation 
facilities  will  be  verified.  The  integrated  test  procedures,  including  control 
procedures,  will  be  exercised  for  the  final  pre-mission  checkout  and  verification. 


Task  Name 

Start 

Finish 

FY96 

FY97 

Qtr  3  Qtr  4 

Qtr  1 

7  Integration  Testing 

6/3/96 

8/1/96 

7.1  Develop  Test  Procedures 

6/3/96 

8/1/96 

■■■ 

7.2  WSSF/SIMLAB  Exch  PDUs 

6/3/96 

6/14/96 

■ 

7.3  WSSF/WSIC  Exch  PDUs 

6/24/96 

6/28/96 

1 

7.4  All  Nodes  Exch  PDUs 

7/1/96 

7/12/96 

a 

7.5  Dry  Runs  w/o  Test  Control 

7/15/96 

7/17/96 

i 

7.6  Dry  Runs  w/  Test  Control 

7/18/96 

7/19/96 

i 

Figure  2.5.2-1.  Integration  Testing  Activities  and  Schedule 


Task  8  TCAC  Preparations.  The  detailed  schedule  for  this  task  is  shown  in  Figure  2.5.2-8. 

The  following  tasks  and  activities  are  indicated  in  the  schedule: 

8.1  Install/Configure  H/W  &  S/W:  Hardware  and  software  needed  for  test  control 
and  monitoring  of  Missions  #2  and  #3  will  be  installed  and/or  configured  in  the 
TCAC.  The  BMIC  will  be  decommissioned  as  the  control  node. 

8.2  Checkout  &  Testing:  The  TCAC  modifications  will  be  verified  during  internal 
testing  at  the  facility 

8.3  Connect  Tl:  The  commercial  T1  between  the  TCAC  and  Pt.  Mugu  will  be  leased 
and  connected. 
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8.4  Baseline  Network  Performance:  The  T1  link  between  the  TCAC  and  Pt.  Mugu 
will  be  tested,  and  the  network  performance  will  be  characterized. 


Task  Name 

Start 

Finish 

FY96 

FY97 

Qtr  4 

Qtri  1  Qtr2 

8  TCAC  Preparations 

8/16/96 

9/27/96 

8.1  Install/Configure  HAV  &  S/W 

8/16/96 

9/5/96 

■3 

8.2  Checkout  &  Testing 

9/3/96 

9/20/96 

■rl 

8.3  Connect  T1 

9/3/96 

9/20/96 

H 

8.4  Baseline  Network  Performance 

9/23/96 

9/27/96 

■H 

Figure  2.5.2-8.  TCAC  Preparations  Activities  and  Schedule 


2.5.3  Test  Conduct  Detailed  Schedule 

The  following  top  level  task  in  Figure  2. 5.1-1  is  a  test  conduct  task: 

Task  9  Test  Missions.  The  detailed  schedule  for  this  task  is  shown  in  Figure  2.5.3-1.  The 
three  missions  are  indicated  in  the  schedule. 


Task  Name 

Start 

Finish 

FY96 

FY97 

Qtr  4 

Qtri  Qtr  2 

9  Test  Missions 

8/5/96 

12/3/96 

9.1  Mission  #1 

8/5/96 

8/29/96 

9.1.1  Test  Execution 

8/5/96 

8/9/96 

1 

9.1.2  Quick-Look  Report 

8/9/96 

8/14/96 

1 

9.1.3  Analysis 

8/9/96 

8/29/96 

■ 

9.2  Mission  #2 

10/7/96 

10/25/96 

m 

9.2.1  Test  Execution 

10/7/96 

10/11/96 

i 

9.2.2  Quick-Look  Report 

T6/1 1/96 

10/16/96 

i 

9.2.3  Analysis 

10/11/96 

10/25/96 

■ 

9.3  Mission  #3 

10/29/96 

12/3/96 

9.3.1  Test  Execution 

10/29/96 

11/1/96 

i 

9.3.2  Quick-Look  Report 

11/1/96 

11/6/96 

i 

9.3.3  Analysis 

. ll/i/96 

12/3/96 

■ 

Figure  2.5.3-1.  Test  Missions  Activities  and  Schedule 


Each  test  mission  consists  of  the  following  activities: 

9.X.  1  Test  Execution:  Each  mission  will  consume  two  days.  The  first  day  will  start 
with  a  dry  run  to  check  connectivity  and  operation  of  all  nodes.  Upon 
successful  completion  of  the  dry  run,  the  actual  mission  will  begin.  Each 
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mission  will  execute  the  corresponding  test  matrix  in  order  to  accomplish  the 
test  objectives  for  the  mission.  Test  control  and  monitoring  will  be  exercised 
from  the  BMIC  in  Mission  #1  and  the  TCAC  in  Missions  #2  and  #3. 

9.X.2  Quick-Look  Report:  A  quick-look  report  will  be  prepared  after  each  mission 
and  will  summarize  performance  of  the  overall  mission. 

9.X.3  Analysis:  Detailed  analysis  will  be  performed  to  verify  that  all  test  objectives 
were  met  and  to  prepare  for  subsequent  missions. 

2.5.4  Post-Test  Detailed  Schedule 

The  following  top  level  task  in  Figure  2.5. 1-1  is  a  post-test  task: 

Task  10  Post-Test  Documentation.  The  detailed  schedule  for  this  task  is  shown  in  Figure 
2.5.4- 1.  Work  on  the  final  report  will  begin  as  soon  as  results  from  Mission  #1  are 
available.  NAWCWPNS  will  provide  JADS  with  a  final  report  after  all  missions 
have  been  completed  and  the  data  analyzed.  JADS  will  combine  the  NAWCWPNS 
report  with  the  results  of  additional  JADS  analysis  for  the  JADS  Final  Report.  The 
final  report  will  be  a  compilation  of  quick-look  and  detailed  analysis,  V&V 
documentation,  lessons  learned,  conclusions,  and  recommendations. 


Task  Name 

Start 

Finish 

FY96 

FY97 

Qtr  4 

Qtrl  1  Qtr2 

10  Post-Test  Documentation 

8/9/96 

3/31/97 

f 

10.1  NAWCWPNS  Final  Report 

8/9/96 

1/2/97 

■■ 

10.2  JADS  Final  Report 

1/2/97 

3/31/97 

■■ 

Figure  2.5.4-1.  Post-Test  Documentation  Activities  and  Schedule 
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3.0  LSP  Execution  Results 


This  section  describes  mission  rehearsal  and  execution  activities,  to  include  conclusions,  with 
emphasis  on  the  deviations  from  the  planned  test  approach. 

3.1  Mission  Rehearsal 

3.1.1  Mission  Rehearsal  Plan 

The  objective  of  the  Mission  Rehearsal  was  to  prepare  for  the  V&V  Mission,  and  the  test  matrix 
was  the  same  as  for  the  V&V  Mission  (Day  1  and  Day  2  runs  in  Table  2.4.2. 1-1).  There  were 
two  sets  of  V&V  trials:  one  set  manual,  the  other  automatic.  The  automatic  trials  were  to  use 
recorded  data  to  replay  a  single  previously  flown  simulation  engagement.  The  manual  runs 
included  trials  with  and  without  flares,  and  the  automatic  runs  were  with  flares  only.  Test  control 
was  exercised  from  the  Battle  Management  Interoperability  Center  (BMIC)  at  Pt  Mugu  CA.  Test 
schedule  slips  due  to  simulator  and  networking  integration  problems  caused  die  Mission  Rehearsal 
period  to  move  into  a  period  originally  scheduled  for  testing  rather  than  rehearsal. 

3.1.2  Mission  Rehearsal  Results 

The  Mission  Rehearsal  took  place  on  27  and  28  August  1996.  Over  the  two  day  period,  107 
trials  were  attempted.  There  were  1 9  manual  runs  with  no  flares,  54  manual  runs  with  flares,  and 
34  automatic  runs  with  flares.  A  trial  was  judged  to  be  completed  if  there  was  a  successful  missile 
launch  and  a  complete  missile  flyout.  A  defined  shot  box  was  established  prior  to  rehearsal 
missions  (Table  4. 1.1. 2-1).  A  trial  could  be  complete  without  being  in  the  shot  box.  Of  the  19 
benign  manual  runs,  14  were  completed,  and  of  those,  9  were  in  the  box  (all  5  runs  which  were 
out  of  the  box  occurred  on  the  first  day  of  testing  when  the  test  controller  and  shooter  pilot  were 
adjusting  simulator  starting  positions  and  shoot  cues  to  achieve  the  LPN-15  launch  conditions). 
Of  the  54  manual  trials  with  flares,  38  were  complete,  and  36  were  in  the  box.  Of  the  34 
automatic  trials  with  flares,  27  were  complete,  and  24  were  in  the  box.  The  automated  runs  did 
not  work  as  planned,  because  the  flight  simulators  were  not  designed  to  support  automatic,  rather 
than  manual,  inputs.  Using  “in  the  box”  as  the  sole  success  criterion,  69  of  the  107,  or  about  65% 
of  the  attempted  trials  were  successful.  From  a  qualitative  perspective,  missile  flyout 
representation  in  the  ADS  environment  was  not  consistent  with  five  missile  behavior.  The  HWIL 
missile  lofted  after  launch  in  the  distributed  environment;  the  real  AIM-9  missile  in  LPN-15  (and 
in  standalone  HWIL  simulations),  under  similar  launch  conditions,  did  not  demonstrate  that 
behavior.  Results  are  summarized  in  Table  3 . 1 .2- 1 . 


Table  3. 1.2-1.  Summary  of  Runs  in  Mission  Rehearsal 


Manual/Auto 

Flares 

In  the  box 

%  in  the  box 

M 

No 

19 

14 

9 

47% 

M 

Yes 

54 

38 

36 

67% 

A 

Yes 

34 

27 

24 

71% 

Total 

107 

79 

69 

64% 
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Reasons  for  not  completing,  or  aborting,  the  runs  included:  WSSF  not  sending  out  PDUs,  shooter 
losing  lock  before  or  at  launch,  WSSF  not  properly  reset,  SIMLAB  not  properly  reset  before  the 
run,  SIMLAB  losing  shooter  PDUs  before  launch,  no  missile  launch,  and  WSSF  not  starting. 

3.1.3  Mission  Rehearsal  Conclusions 

-  Test  procedures,  including  control  procedures,  worked  very  well.  The  test  controller  and 
pilots  were  able  to  achieve  reproducible  launch  conditions  within  a  minimal  number  of  trial 
runs.  However,  step  calls  were  deemed  necessary  by  NAWCWPNS  for  the  remaining 
missions  due  to  OPSEC  concerns.  The  step  calls  replaced  certain  actions  directed  by  the 
test  controller  during  the  runs  with  step  numbers. 

The  automatic  replay  runs  were  unable  to  achieve  their  objective:  precise  replication  of 
launch  conditions  and  target  trajectory.  This  resulted  because  several  manual  actions  were 
required:  manual  and  independent  starts  of  the  WSSF  and  WSIC,  manual  trigger  squeeze 
by  the  shooter  pilot,  and  manual  initiation  of  the  flare  simulator.  As  a  result,  the  automatic 
replay  runs  were  removed  from  the  V&V  Mission  and  Parametric  Study  Mission  test 
matrices. 

-  The  manual  V2  runs  produced  sufficient  data  for  evaluating  the  reproducibility  of  profile 
conditions  by  the  pilots.  This  is  the  major  consideration  in  achieving  Test  Objective  2. 

-  The  simulated  missile  flyout  in  the  SIMLAB  was  judged  to  be  anomalous  by  the  AIM-9 
expert.  The  shooter  was  about  900  feet  higher  in  altitude  than  the  target,  and  the  missile 
should  have  steadily  lost  altitude  during  its  flyout.  Instead,  the  missile  “lofted”  to  an 
altitude  above  the  shooter  before  losing  altitude  and  guiding  to  the  target. 

Preparation  for  testing  often  takes  longer  than  planned.  (Stated  another  way,  planners 
have  a  tendency  to  underestimate  the  schedule  requirements  for  set-up  and  checkout.) 

-  Test  rehearsal  periods  need  to  be  followed  by  a  lab  or  experimental  period,  during  which 
fixes  and  corrections  (for  things  like  the  lofting  problem)  are  explored  in  a  linked 
environment.  An  experimental  period  was  conducted  on  22  Oct  to  try  to  fix  the  lofting 
problem.  Many  fixes  cannot  be  satisfactorily  explored  in  unlinked  conditions. 

3.2  V&V  Mission 

3.2.1  V&V  Mission  Plan 

The  primary  objective  of  the  V&V  Mission  was  to  accomplish  V&V  of  the  LSP  configuration, 
and  most  of  the  runs  were  dedicated  to  that  purpose.  The  V&V  runs  followed  the  manual  runs 
for  the  V&V  Mission  test  matrix  (Day  1  runs  in  Table  2.4.2.1-1).  A  second  set  of  modified  trials 
were  specifically  aimed  at  investigating  the  missile  lofting  problem  discovered  in  the  August 
rehearsals.  The  modified  runs  explored  flyout  behavior  when  the  shooter  nose  was  above  or 


3-2 


below  the  original  (essentially  boresight)  test  conditions  at  launch.  A  final  set  of  trials  were 
performed  to  verify  the  ability  of  the  WSIC  to  generate  a  FIRE  (Flare)  PDU  immediately  upon 
receipt  of  the  missile  launch  cue  and  the  ability  of  the  SIMLAB  to  use  the  FIRE  (Flare)  PDU  to 
initiate  the  flare  simulator  (profile  P3  from  the  Parametric  Study  Mission,  Table  2.4.2.2-1). 
Because  of  Mission  Rehearsal  results,  automated  runs  were  eliminated  from  the  test,  and  the  test 
period  was  reduced  from  two  days  to  one  day.  The  modified  test  matrix  is  given  in  Table  3. 2.1-1. 
Test  control  was  exercised  from  the  Test  Control  and  Analysis  Center  (TCAC)  in  Albuquerque, 
NM. 


Table  3.2.1-1.  Modified  V&V  Mission  Test  Matrix 


Profile/ 

Repetitions 

Flare  Configuration 

Remarks 

No 

Flares 

Manual 

Flares 

Auto 

Flares 

Vl/10 

X 

Verification:  Manual  replicate  live  fire  profile 
without  flares 

V2/20 

X 

Validation:  Manual  replicate  live  fire  profile 
(LPN-15) 

Modified  V2/2 

X 

Lofting  Study:  Shooter  nose  above  boresight 
and  to  left 

Modified  V2/2 

X 

Lofting  Study:  Shooter  nose  above  boresight 
and  to  right 

Modified  V2/2 

X 

Lofting  Study:  Shooter  nose  below  boresight 
and  to  left 

Modified  V2/2 

X 

Lofting  Study:  Shooter  nose  below  boresight 
and  to  right 

P3/2 

X 

Verify  ability  to  execute  Parametric  Mission 
runs 

3.2.2  V&V  Mission  Results 

The  V&V  Mission  was  performed  on  29  October  1996.  There  were  6  manual  VI  runs  without 
flares,  all  were  complete,  and  4  were  in  the  box.  There  were  21  manual  V2  runs  with  flares.  Of 
those,  13  were  complete,  and  10  were  in  the  box.  There  were  6  modified  manual  runs,  with 
flares,  designed  to  explore  the  lofting  problem.  Of  those,  only  two  were  complete,  but  the  other 
four  contained  enough  of  the  initial  flyout  data  to  support  the  lofting  investigation.  Finally,  there 
were  3  manual  P3  runs  to  verify  automatic  flare  initiation.  Of  those,  only  one  was  complete  with 
successful  flare  initiation.  Excluding  the  lofting  experiment  and  automatic  flare  initiation  runs 
(these  did  not  attempt  to  launch  in  the  box),  14  of  27  V&V  runs  were  in  the  box  for  an  apparent 
success  rate  of  approximately  52%.  Results  for  the  V&V  runs  are  summarized  in  Table  3.2.2-1 

Reasons  for  not  completing,  or  aborting,  the  V&V  runs  included:  WSIC  not  running  and 
SIMLAB  not  properly  reset  before  run.  During  the  automatic  flare  initiation  runs,  the  first 
attempt  did  not  initiate  the  flares  because  the  SIMLAB  code  had  not  been  switched  to  allow  FIRE 
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(Flare)  PDU  initiation  of  the  flares  on  the  first  run.  This  was  fixed  for  the  second  and  third 
attempts.  However,  the  second  attempt  was  aborted  because  the  shooter  was  not  seeing  the 
target. 

Table  3.2.2-1.  Summary  of  V&V  Runs  in  V&V  Mission 


3.2.3  V&V  Mission  Conclusions 

-  The  V&V  objective  could  not  be  fully  accomplished.  The  missile  lofting  problem  had  not 
been  fixed,  so  that  the  missile  flyouts  were  still  judged  to  be  invalid. 

-  The  lofting  investigation  showed  that  lofting  still  occurred  even  when  the  shooter’s  nose 
was  pointed  well  below  the  target.  Hence,  it  was  concluded  that  the  shooter  attitude  at 
launch  was  not  causing  or  contributing  to  the  lofting. 

The  target  latitude  received  at  the  SIMLAB  was  compared  to  that  computed  in  the  missile 
simulation  by  integrating  the  target’s  north  velocity  component.  The  result  was  that  the 
simulation  computed  a  target  location  that  steadily  diverged  from  the  value  received 
directly  from  the  WSIC.  The  largest  value  of  divergence  was  always  at  the  end  of  the 
missile  flyout  and  ranged  from  150  to  300  feet  (the  latitude  computed  by  the  missile 
simulation  was  north  of  the  value  from  the  WSIC  by  these  amounts). 

-  Latency  values  between  nodes  varied  significantly;  the  values  were  not  statistically  well 
behaved.  Latency  values  were  largely  associated  with  processing,  to  include  buffering, 
delays.  Latency  values  between  the  creating  simulation  and  the  NIU  at  the  creating 
simulation  node  varied  from  10  milliseconds  to  over  one  second.  (The  collected  data  were 
used  to  support  the  latency  test  objective,  Test  Objective  3 .) 

-  The  simulation  computer  at  the  SIMLAB  overheated  and  quit.  The  cause  of  the 
overheating  appeared  to  be  the  floodlights  attached  to  the  ceiling  for  videotaping. 

3.3  Parametric  Study  Mission 

3.3.1  Parametric  Study  Mission  Plan 

Because  of  the  lofting  problem  experienced  during  the  V&V  Mission,  some  of  the  trials  in  the 
Parametric  Study  Mission  had  to  be  devoted  to  V&V  activity.  These  runs  followed  the  manual 
runs  for  the  V&V  Mission  test  matrix  (Day  1  runs  in  Table  2.4.2. 1-1).  Additional  runs  were  to 
follow  a  subset  of  the  original  Parametric  Study  test  matrix  (Table  2.4.2.2-1).  The  original  test 
matrix  contained  ten  trials  for  each  parametric  case  (profiles  P1-P8  in  Table  2.4.2.2-1),  5  manual, 
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and  5  automatic  over  two  days  (80  runs  total).  The  test  matrix  was  modified  to  40  manual  runs, 
and  a  single  day,  because  of  the  problem  with  automated  runs  cited  earlier.  The  modified  test 
matrix  is  given  in  Table  3.3. 1-1.  There  were  to  have  been  20  runs  of  the  base  case  V&V  flare 
profile  (P2)  and  4  sets  of  5  runs  as  follows:  one  set  of  the  base  case  V&V  non-flare  profile  (PI), 
two  sets  with  automatic  flare  dispense  at  +0  and  +4  seconds  warning  cue  delay  (P3  and  P5),  and 
one  set  with  automatic  flare  dispense  at  +4  seconds  and  a  7g  turn  vice  the  3.6g  turn  used  in  all 
other  trial  sets  (P8).  Test  control  was  exercised  from  the  TCAC  at  Albuquerque. 


Table  3.3.1-1.  Modified  Parametric  Study  Mission  Test  Matrix 


Profile/ 

Repetitio 

ns 

VARIABLE  PARAMETER 

REMARKS 

Flare  Dispense  Time 

Target  Maneuver 
Time 

N/ 

C 

+0 

s 

+2 

s 

+4 

s 

N/ 

C 

+0 

s 

+2 

s 

+4 

s 

Pl/5 

■ 

■ 

■ 

X 

■ 

Baseline  verification  profile  (VI) 

No  flares 

P2/20 

X 

■ 

X 

■ 

■ 

Baseline  validation  profile  (V2) 

No  warning  cue 

P3/5 

■ 

X 

X 

■ 

Automatic  flare  dispense 

Warning  cue  at  +0  sec 

P5/5 

■ 

X 

X 

■ 

Automatic  flare  dispense 

Warning  cue  at  +4  sec 

P8/5 

■ 

X 

■ 

■ 

X 

Automatic  flare  dispense 

Maneuver:  increase  turn  to  7+  g 
Warning  cue  at  +4  sec 

Note:  N/C  =  no  change  from  baseline  (V&V)  profile 


3.3.2  Parametric  Study  Mission  Results 

The  Parametric  Study  Mission  was  performed  on  19  November  1996.  Only  26  trials  were 
attempted  versus  the  40  in  the  modified  plan,  largely  because  the  AIM-9  HWIL  missile  went 
down  for  about  an  hour.  Profiles  identical  to  the  non-flare  and  flare  V&V  profiles  (PI  and  P2  in 
Table  3. 3. 1-1),  with  no  launch  warning  cues,  were  used  for  25  of  the  runs.  Thus,  these  trials 
served  to  support  the  V&V  test  objective.  Of  5  manual  non-flare  trials,  3  were  complete,  and  2 
were  in  the  box.  Of  20  manual  flare  trials,  6  were  complete,  and  all  6  were  in  the  box.  One 
additional  trial,  a  parametric  trial  with  a  warning  cue  at  launch  (P3),  was  attempted,  but  not 
completed.  The  in-the-box  success  rate  was  only  31%. 

Reasons  for  not  completing,  or  aborting,  the  runs  included:  no  target  PDUs,  lack  of  pilot  control 
of  the  WSIC,  WSIC  and  WSSF  not  running,  hung  missile,  SIMLAB  NIU  failure,  SIMLAB  not 
properly  reset  for  run,  no  lock  on  target,  and  no  shoot  cue. 
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3.3.3  Parametric  Study  Mission  Conclusions 

-  Sufficient  data  for  V&V  were  obtained.  The  missile  lofting  problem  was  fixed,  and  only  a 
single  complete  run  “in  the  box”  was  required  for  validation  of  the  missile  simulation. 

-  The  lofting  problem  appeared  to  be  caused  by  a  sign  err  or  on  the  vertical  velocity 
component  of  the  shooter  simulation  which  led  the  HWIL  missile  to  believe  the 
shooter  was  climbing,  when  it  was  actually  descending. 

-  The  target  “latitude  divergence”  was  still  evident,  but  was  significantly  reduced  from  the 
V&V  Mission.  The  reduction  appeared  to  be  due  to  correcting  the  SIMLAB  conversion 
between  its  internal  inertial  coordinates  (local  tangent  plane)  and  geodetic  coordinates 
(latitude,  longitude,  altitude)  and  to  increasing  the  iteration  rate  of  the  target  entity  state 
PDUs  at  the  WSIC  NIU  (40  Hz  versus  20  Hz). 

-  Failure  of  the  AIM-9  seeker  at  the  SIMLAB  resulted  in  extended  downtime  and  prevented 
the  accomplishment  of  the  entire  Parametric  Mission  test  matrix. 

-  Latency  magnitudes  and  variability  were  both  significantly  reduced  in  this  set  of  trials,  to 
30-60  milliseconds  between  the  originating  simulation  and  the  NIU  at  the  receiving  node, 
apparently  because  the  NIUs  were  reset  before  each  run.  Data  from  this  mission  were  also 
used  to  accomplish  Test  Objective  3. 

-  Linking  tends  to  highlight  areas  of  simulation/simulator  performance  which  are  incorrect. 

-  Although  limited  data  were  gathered  to  support  the  originally  intended  parametric  studies, 
the  trial  data,  in  total,  suggests  the  ADS  architecture  has  utility  for  parametric  studies. 

3.4  Latency  Study  Mission: 

Originally,  the  test  team  envisaged  a  series  of  trials  where  latency  would  be  treated  as  a  controlled 
variable,  and  the  sensitivity  of  measures  of  interest  to  latency  values  could  be  quantitatively 
explored  (Table  2.4.2.3-1).  Because  of  a  combination  of  schedule  slips,  and  an  abundance  of 
latency  data  collected  in  the  missions  already  performed,  the  Latency  Study  Mission  was  judged 
to  be  redundant,  and  it  was  canceled. 
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4.0  Analysis  of  Test  Objectives 

4.1  Test  Objective  1:  Assess  the  validity  of  AIM-9  data  obtained  in  the  LSP  ADS 
configuration 

This  objective  involves  both  verification  and  validation  of  the  LSP  results.  Verification  is  the 
process  of  determining  that  a  simulation  accurately  represents  the  developers  ’  conceptual 
description  and  specifications.  Validation  is  the  process  of  determining  the  extent  to  which  the 
simulation  accurately  represents  the  real  world  from  the  perspective  of  the  intended  uses. 

4.1.1  Verification 

The  WSSF,  WSIC,  and  SIMLAB  have  been  previously  verified  for  their  intended  uses. 
Therefore,  the  LSP  verification  is  to  confirm  that  the  simulation  (i.e.,  the  three  linked  simulators) 
operates  as  designed  when  the  simulators  are  real-time  linked  through  an  ADS  network  and  can 
be  used  to  replicate  an  actual  live  fire  test,  including  the  necessary  data  collection.  The 
verification  analysis  also  determined  the  fidelity  of  entity  state  data  transfer  between  the 
simulation  nodes. 

4.1.1.1  Verification  Test  Method 

Data  for  verification  were  collected  during  integration  testing  and  dry  runs,  in  addition  to  data 
from  executing  the  missions. 

The  qualitative  verification  involved  demonstrating  that  the  simulators  are  properly  linked  by  their 
behavior  during  integration  testing  and  dry  runs,  as  well  as  the  actual  missions.  During  the 
missions,  the  first  several  passes  (5-10)  were  used  for  verification  and  used  the  basic  five  fire 
profile,  but  without  flare  countermeasures  (verification  passes  were  designated  as  Profile  VI). 
Successful  qualitative  verification  resulted  if  the  LSP  inputs  were  demonstrated  to  correctly  drive 
each  simulator  -  i.e.  the  F/A-18  WSSF  acquired  and  tracked  the  F-14  WSIC;  the  WSSF  provided 
the  proper  prelaunch  inputs  to  the  AIM-9  SIMLAB  and  launched  the  missile;  the  SIMLAB 
responded  to  the  WSSF  prelaunch  and  launch  initiate  inputs;  the  SIMLAB  acquired  and  tracked 
the  WSIC;  and  the  WSIC  acted  as  the  target. 

The  quantitative  verification  involved  checking  the  flow  of  entity  state  data  between  the 
simulation  nodes  during  the  actual  missions.  Where  possible,  data  were  collected  at  four  check 
points  as  they  were  created  by  one  simulator,  processed,  transmitted  to,  and  received  by  another 
simulator.  Figure  4.1. 1.1-1  shows  the  generic  data  acquisition  points.  The  output  of  the 
originating  simulator  “A”  (i.e.,  its  generated,  “raw”  data)  was  recorded  at  (1),  the  originating 
simulation  computer.  The  PDU  output  of  the  NIU  at  the  originating  simulation  facility  was 
recorded  at  (2),  the  PDU  logger  in  the  originating  facility.  The  PDUs  received  at  the  NIU  at  the 
receiving  simulation  facility  was  recorded  at  (3),  the  PDU  logger  in  the  receiving  facility.  FinaUy, 
the  data  input  to  the  receiving  simulator  “B”  was  recorded  at  (4),  the  receiving  simulation 
computer. 
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Figure  4.1.1.1-1.  Data  Collection  for  Quantitative  Verification 


4.1. 1.2  Verification  Analysis  Method 

The  qualitative  verification  analyses  involved  verifying  the  following: 

-  The  WSSF  and  WSIC  executed  their  respective  profiles  and  achieved  the  desired  shot  box 
parameters  given  in  Table  4.1. 1.2-1.  This  was  determined  from  the  quick-look  readout  of 
the  launch  conditions  in  the  SIMLAB. 


Table  4.1.1.2-1.  Shot  Box  for  LSP  Trials 


Parameter 

Acceptable  Range 
(Relative  to  LPN-15  Value) 

Velocity  of  Each  Aircraft  at  Launch 

±50  ft/s 

Altitude  of  Each  Aircraft  at  Launch 

±500  ft 

Launch  Range 

±1500  ft 

Shooter  Lead  Angle 

±5° 

Target  Aspect  at  Launch 

±10° 

-  The  WSSF  launched  the  missile  in  the  SIMLAB.  This  occurred  when  trigger  squeeze  by 
the  WSSF  pilot  resulted  in  initiation  of  the  SIMLAB  missile  HWIL  simulation. 

-  The  missile  in  the  SIMLAB  responded  to  the  target  during  its  flyout: 

-  The  missile  seeker  tracked  the  target,  as  determined  by  AIM-9  expert  monitoring  the 
seeker  telemetry  (TM)  channels. 

-  Quick-look  missile  TM  channels  in  SIMLAB  were  operating  properly.  This  was  verified 
by  the  SIMLAB  operators. 

-  All  data  recording  systems  were  operating  properly.  This  was  verified  by  reports  from 
each  facility. 

The  quantitative  verification  involved  analysis  of  the  entity  state  data  (position,  velocity,  and 
orientation  of  entity)  as  follows: 
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-  For  each  pair  of  simulators,  data  collected  at  points  (1)  and  (2)  in  Figure  4.1. 1.1-1  were 
compared  to  determine  if  the  conversion  of  high-fidelity  simulation  output  to  PDUs 
occurred  correctly  (format  and  accuracy).  The  entity  state  data  undergo  a  reference  frame 
transformation  when  PDUs  are  created,  so  that  the  entity  state  data  elements  could  not  be 
directly  compared.  Instead,  the  appropriate  transformations  were  applied  independently  to 
the  PDU  data  before  comparing  the  simulation  output.  Differences  between  the 
transformed  PDU  data  elements  and  the  simulation  output  were  computed.  Non-zero 
differences  were  noted,  but  were  partly  due  to  errors  in  independently  transforming  the 
PDU  data  for  the  comparison. 

Data  collected  at  points  (2)  and  (3)  in  Figure  4.1. 1.1-1  were  compared  to  determine  if  the 
data  were  properly  transmitted  between  facilities.  This  was  a  direct  comparison  between 
PDU  data  elements  collected  at  each  facility.  Differences  between  the  data  elements  of 
the  PDUs  were  computed.  Non-zero  differences  were  attributed  to  errors  in  the 
transmission  process  (ADS-induced  errors). 

Data  collected  at  points  (3)  and  (4)  in  Figure  4.1. 1.1-1  were  compared  to  determine  if  the 
conversion  of  PDUs  to  high-fidelity  simulation  input  occurred  correctly.  As  above,  the 
appropriate  transformations  were  applied  independently  to  the  PDU  data  before 
comparing  the  simulation  output.  Differences  between  the  transformed  PDU  data 
elements  and  the  simulation  input  are  computed.  Non-zero  differences  were  noted,  but 
were  partly  due  to  errors  in  independently  transforming  the  PDU  data  for  the  comparison. 

Data  collected  at  points  (1)  and  (4)  in  Figure  4.1. 1.1-1  we  re  directly  compared  to 
determine  the  net  effect  on  data  transfer  between  simulation  nodes. 

The  comparison  was  done  by  matching  data  recorded  at  each  pair  of  facilities  for  each 
data  element  (e.g.,  the  latitude  position  of  an  entity  generated  at  one  facility  was  received 
by  another  facility). 

Static  Verification .  The  direct  matching  of  entity  state  data  was  done  by  freezing  the  entity  in 
the  originating  simulation  so  that  the  same  set  of  data  was  repeatedly  created  and  transmitted  to 
the  other  facilities.  This  was  necessary  because  the  positional  data  out  of  or  into  the  shnulations 
was  dead  reckoned  at  the  NIUs  during  dynamic  operation,  preventing  a  direct  comparison.  The 
full  four-point  comparison  (Fig.  4. 1.1. 1-1)  could  only  be  performed  for  data  exchanged  between 
the  target  (F-14  WSIC)  and  the  shooter  (F/A-18  WSSF). 

The  four-point  comparison  did  not  work  for  the  shooter  or  target  data  sent  to  the  SIMLAB.  The 
SIMLAB  only  logs  raw  data  when  the  AIM-9  simulation  is  actually  running  (during  missile 
flyout).  As  a  result,  the  raw  shooter  and  target  data  produced  when  the  WSSF  and  WSlO  were 
“frozen”  were  not  logged  at  the  SIMLAB  and  could  not  be  used  in  the  verification  process. 

Dynamic  Verification  .  The  verification  methodology  outlined  in  the  LSP  TAP  needed  revision  in 
order  to  verify  the  target  data  being  transmitted  to  the  SIMLAB  during  the  missile  flyout.  Since 
the  target  position  was  changing  with  time  during  the  flyout,  a  dynamic  (F-14  simulator  flying), 
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rather  than  static  (F-14  simulator  “frozen”),  comparison  was  needed.  This  was  attempted  by 
comparing  the  time  histories  of  the  target  latitude,  longitude,  altitude  derived  from  (1)  raw  data 
logged  in  the  WSIC  simulation,  (2)  PDUs  logged  at  the  WSIC,  (3)  PDUs  logged  at  the  SIMLAB, 
and  (4)  raw  data  logged  in  the  SIMLAB  simulation.  It  was  hoped  that  the  time  histories  would  all 
have  the  same  shape,  and  simply  be  displaced  in  time  due  to  latency  between  the  logging 
locations.  However,  there  were  problems  in  directly  comparing  the  time  histories: 

-  The  raw  data  logged  in  the  WSIC  simulation  were  not  logged  during  the  actual  LSP  runs, 
but  by  replaying  the  WSIC  simulation  later  and  sampling  the  data.  As  a  result,  the 
sampled  data  points  represented  different  times  than  those  from  the  PDUs  logged  at  the 
WSIC  (see  Fig.  4.1. 1.2-1).  This  prevented  a  direct  comparison  between  (1)  and  (2) 
above.  Instead,  the  static  results  were  used  to  estimate  the  accuracy  of  transforming  the 
raw  WSIC  simulation  data  into  PDUs,  and  the  PDUs  created  at  the  WSIC  NIU  were 
assumed  to  represent  the  raw  simulation  output,  within  the  accuracy  of  the  static  results. 


Figure  4.1.1.2-1.  Target  Latitude  vs.  Time  During  Run  #23  (10/29/96) 

-  When  the  target  PDU  data  logged  at  the  SIMLAB  (according  to  log  time)  were  compared 
to  the  PDU  data  logged  at  the  WSIC  (according  to  PDU,  or  creation,  time),  the  result  was 
a  target  trajectory  curve  in  which  the  individual  data  points  were  “misaligned”  in  time  (see 
Fig.  4. 1.1. 2-1).  In  other  words,  the  trajectory  curve  went  from  smooth  in  shape  at  the 
WSIC  to  “unsmoothed”  in  shape  at  the  SIMLAB.  This  was  caused  by  variations  in  the 
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WSIC-to-SIMLAB  latency  and  prevented  a  direct  comparison  between  (2)  and  (3)  above. 

If  the  latency  had  been  constant,  the  SIMLAB  trajectory  would  have  had  the  same  shape 
as  the  WSIC  trajectory,  but  delayed  in  time  by  a  fixed  amount  (the  latency  value). 

-  The  raw  target  data  logged  in  the  SIMLAB  simulation  corresponded  to  different  times 
than  that  from  the  PDU  data  logged  at  the  SIMLAB  (see  Fig.  4. 1.1. 2-1).  Note  that  the 
data  were  read  into  the  SIMLAB  simulation  at  a  higher  rate  than  the  logged  PDU  data  and 
that  dead  reckoning  was  used.  This  prevented  a  direct  comparison  between  (3)  and  (4) 
above. 

The  result  of  these  problems  was  that  the  verification  methodology  outlined  in  the  LSP  TAP  had 
to  be  modified  as  follows: 

-  As  Figure  4. 1.1. 2-1  shows,  latency  affected  the  transfer  of  dynamic  entity  state  data  from 
the  WSIC  to  the  SIMLAB.  In  order  to  quantify  the  effect  on  data  validity,  the  latency 
between  the  WSIC  and  SIMLAB  simulations  was  analyzed  to  determine  the  mean  and 
standard  deviation  values. 

—  The  latency  was  computed  as  follows: 

—  The  target’s  north  component  of  velocity  from  a  target  entity  state  PDU  was 
matched  to  that  from  a  SIMLAB  raw  simulation  frame.  Due  to  the  target’s 
motion,  this  component  of  velocity  exhibited  the  most  change  between  frames  (i.e., 
the  maximum  target  acceleration  is  in  the  north  direction  during  the  engagement) 
and  could  be  most  readily  matched.  The  NIUs  did  not  dead  reckon  velocity,  so 
that  a  direct  comparison  was  possible. 

—  The  latency  was  computed  from  the  difference  in  the  SIMLAB  frame  time 
(determined  by  the  IRIG  times  for  the  frames)  and  the  PDU  time  (which  gives  the 
time  that  the  data  were  created  in  the  WSIC  simulation). 

—  The  standard  deviation  represented  a  random  component  of  the  latency  and  resulted  in 
a  variation  in  the  time  that  the  target  entity  state  data  were  received  at  the  SIMLAB. 
Since  the  SIMLAB  simulation  ran  asynchronously  and  in  real-time,  these  time 
variations  resulted  in  position  variations,  or  uncertainties,  in  the  simulation  (data  were 
fed  into  the  simulation  as  they  were  received,  without  reference  to  their  creation  time). 

-  The  standard  deviation  of  the  latency  was  used  to  estimate  the  uncertainty  in  the  target 
location  perceived  at  the  SIMLAB  compared  to  the  value  generated  by  the  WSIC.  The 
uncertainty  in  the  target  position  due  to  the  random  latency  variations  was  estimated  by 
multiplying  the  standard  deviation  of  the  latency  by  the  target  velocity: 

8x  =  v  fit  0) 

where  fix  =  uncertainty  in  target  position 
v  =  target  velocity 

fit  =  standard  deviation  of  the  WSIC-SIMLAB  latency 
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4.1.1.3  Verification  Results 


Qualitative  verification  before  the  Mission  Rehearsal  indicated  that: 

-  The  shooter  was  tracking  the  target. 

The  shooter  was  initializing  and  launching  the  missile. 

-  The  HUD  display  in  the  SIMLAB  replicated  the  HUD  display  in  the  WSSF. 

-  The  missile  was  tracking  and  intercepting  the  target. 

Verification  of  F/A-18  Data  Between  WSSF  and  WSIC  .  The  F/A-18  entity  state  data  were 
checked  according  to  the  four-point  comparison  illustrated  in  Figure  4.1. 1.1-1  when  the  WSSF 
simulation  was  “frozen”  (static  verification).  The  results  for  positional  data  are  given  in  Table 
4.1. 1.3-1  which  gives  the  differences  (deltas)  in  latitude,  longitude,  and  altitude  from  each 
previous  logging  location  (latitude  and  longitude  differences  were  directly  measured  and 
converted  into  distances  in  the  table).  Note  that  the  actual  PDU  data  were  not  in  the  form  of 
latitude,  longitude,  and  altitude  and  had  to  be  converted  into  this  form  for  the  comparison.  The 
conversion  was  done  using  TCAC  algorithms. 


Table  4.1.1.3-1.  Verification  of  Shooter  Positional  Data  Passed  From  WSSF  to  WSIC 


Location 

Latitude 

Longitude 

Altitude 

Delta  Lat 
(deg) 

Delta  Lat 
(«) 

Delta  Lon 
(deg) 

Delta  Lon 

(ft) 

Delta  Alt 

(ft) 

WSSF  raw 

35.70416638 

-117.6888887 

11700.00 

WSSF  logger 

35.70416583 

-117.6888842 

11699.45 

-5.5E-07 

-0.201 

4.5E-06 

1.335 

-0.55 

WSIC  logger 

35.70416583 

-117.6888842 

11699.45 

0 

0 

0 

0 

0 

WSIC  raw 

35.70416600 

-117.6888810 

11699.41 

1.7E-07 

0.062 

3.2E-06 

0.95 

-0.037 

Net  Deltas 

-3.8E-07 

-0.139 

7.7E-06 

2.285 

-0.587 

Total  of  Net 
Deltas  (ft) 

2.36 

The  shooter  velocity  components  (north,  east,  down)  and  orientation  components  (roll,  pitch, 
yaw)  were  also  checked  according  to  the  four-point  comparison.  These  were  all  found  to  agree 
with  each  other  within  the  precision  of  the  raw  data  readout. 


The  conclusions  are: 

-  The  positions  from  the  PDUs  agreed  with  the  WSSF  raw  data  to  within  1.5  ft  (root-sum- 
square  (RSS)  of  the  deltas  from  the  second  row  of  Table  4.1.13-1).  This  reflected  the 
accuracy  of  the  TCAC  algorithms  for  converting  the  PDU  coordinates  into  latitude, 
longitude,  and  altitude,  as  well  as  the  accuracy  of  the  lab  frame-to-PDU  frame  conversion. 

-  The  PDU  data  were  not  changed  during  transmission  from  the  WSSF  to  the  WSIC. 

-  The  total  net  accuracy  for  transmitting  target  positional  data  from  the  WSSF  to  the  WSI  C 
(via  PDUs)  is  about  2.4  ft  (the  total  was  obtained  from  the  RSS  of  the  individual  nets), 
and  was  acceptable  for  the  LSP  scenario.  Note  that  the  comparison  of  raw  simulation 
data  were  direct  and  did  not  require  coordinate  transformations. 


-  The  shooter  velocity  and  orientation  data  were  not  changed  significantly  during 
transmission  from  the  WSSF  simulation  to  the  WSIC  simulation  (values  agreed  within  the 
precision  of  the  raw  data  readout). 

Verification  of  F-14  Data  Between  WSIC  and  WSSF  .  The  F-14  entity  state  data  were 
checked  according  to  the  four-point  comparison  illustrated  in  Figure  4.1. 1.1-1,  and  results  are 
given  in  Table  4. 1.1. 3-2  (latitude  and  longitude  differences  were  directly  measured  and  converted 
into  distances  in  the  table).  As  above,  the  actual  PDU  data  were  converted  into  latitude, 
longitude,  and  altitude  for  the  comparison. 


Table  4.1.1.3-2.  Verification  of  Target  Positional  Data  Passed  From  WSIC  to  WSSF 


Location 

Latitude 

Longitude 

Altitude 

Delta  Lat 
(deg) 

Delta  Lat 

(ft) 

Delta  Lon 
(deg) 

Delta  Lon 

(ft) 

Delta  Alt 

(ft) 

WSIC  raw 

35.727818 

-117.697784 

10392.5 

WSIC  logger 

35.72781809 

-117.6977811 

10392.17 

9E-08 

0.033 

2.9E-06 

0.860 

-0.328 

WSSF  logger 

35.72781809 

-117.6977811 

10392.17 

0 

0 

0 

0 

0 

WSSF  raw 

35.72781808 

-117.6977787 

10393.1 

-IE-08 

-0.004^ 

2.4E-06 

0.712 

0.928 

Net  Deltas 

8E-08 

0.029 

5.3E-06 

1.572 

0.6 

Total  of  Net 
Deltas  (ft) 

1.68 

The  target  velocity  components  (north,  east,  down)  and  orientation  components  (roll,  pitch,  yaw) 
were  also  checked  according  to  the  four-point  comparison.  These  were  all  found  to  agree  with 
each  other  within  the  precision  of  the  raw  data  readout. 

The  conclusions  are: 

-  The  positions  from  the  PDUs  agree  with  the  WSIC  raw  data  to  within  1  ft  (RSS  of  deltas 
from  second  row  of  Table  4. 1.1. 3-2).  This  reflects  the  accuracy  of  the  TCAC  algorithms 
for  converting  the  PDU  coordinates  into  latitude,  longitude,  altitude,  as  well  as  the 
accuracy  of  the  lab  frame-to-PDU  frame  conversion. 

-  The  PDU  data  were  not  modified  during  transmission  from  the  WSIC  to  the  WSSF. 

-  The  total  net  accuracy  for  transmitting  target  positional  data  from  the  WSIC  to  the  WSSF 
(via  PDUs)  is  about  1.6  ft  (the  total  was  obtained  from  the  RSS  of  the  individual  nets), 
and  was  acceptable  for  the  LSP  scenario. 

-  The  target  velocity  and  orientation  data  were  not  changed  significantly  during  transmission 
from  the  WSIC  raw  simulation  to  the  WSSF  raw  simulation  (values  agreed  within  the 
precision  of  the  raw  data  readout). 

-  Comparing  Table  4. 1.1. 3-2  with  Table  4.1.  1.3-1  shows  smaller  net  differences  for  the 
target  latitude  and  longitude.  The  reason  for  these  smaller  differences  is  not  known. 
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Verification  of  F/A-18  Data  Between  WSSF  and  SIMLAB  .  The  SIMLAB  only  logs  raw  data 
when  the  AIM-9  simulation  is  actually  running  (during  missile  flyout).  As  a  result,  the  raw 
shooter  (F/A-18  WSSF)  data  produced  when  the  WSSF  was  “frozen”  were  not  logged  at  the 
SIMLAB  and  could  not  be  used  in  the  verification  process.  Instead,  the  results  of  the  verification 
of  the  shooter  data  between  the  WSSF  and  the  WSIC  (Table  4.1. 1.3-1)  were  assumed  to  also 
apply  to  the  transfer  between  the  WSSF  and  the  SIMLAB.  Also,  the  launch  conditions 
determined  at  both  the  WSSF  and  at  the  SIMLAB  were  compared  and  were  found  to  agree  within 
acceptable  tolerances  (see  results  for  Test  Objective  3  below).  Note  that  the  only  shooter  entity 
state  data  used  by  the  SIMLAB  missile  simulation  were  the  shooter  initial  conditions. 

Verification  of  PDU  Data  Between  All  Nodes  .  Spot  checks  were  performed  on  the  PDUs  for 
all  three  entities  by  comparing  entity  state  values  for  the  same  PDU  time  from  PDUs  logged  at 
different  nodes.  In  all  cases,  the  entity  state  data  matched  perfectly. 

Verification  of  F-14  Data  Between  WSIC  and  SIMLAB  .  As  noted  for  the  F/A-18  data,  the 
raw  target  (F-14  WSIC)  data  produced  when  the  WSIC  was  “frozen”  were  not  logged  at  the 
SIMLAB  and  could  not  be  used  in  the  verification  process.  Instead,  the  raw  data  produced  at  the 
WSIC  could  only  be  compared  to  PDU  data  logged  at  the  SIMLAB.  The  F-14  PDUs  logged  at 
the  SIMLAB  exactly  matched  those  logged  at  the  WSIC,  so  that  the  results  for  the  WSIC/WSSF 
comparison  apply: 

-  The  positions  from  the  PDUs  agreed  with  the  WSIC  raw  data  to  within  1  ft. 

-  The  PDU  data  were  not  modified  during  transmission  from  the  WSIC  to  the  SIMLAB. 

The  target  data  being  transmitted  to  the  SIMLAB  during  the  missile  flyout  was  verified  by 
applying  the  dynamic  verification  process  outlined  in  Section  4. 1.1. 2.  Equation  1  was  used  to 
estimate  the  uncertainty  in  the  target  position  data  input  into  the  SIMLAB  simulation.  Data  from 
the  Parametric  Study  Mission  runs  (11/19/96)  were  used,  because  these  runs  exhibited  the 
smallest  latencies  of  any  mission.  Results  are  given  in  Table  4.1. 1.3-3. 


Table  4.1.1.3-3.  Uncertainty  in  Target  Position  Input  into  SIMLAB  Simulation 


Run  # 
(11/19/96) 

Total  Latency  (ms) 

Target 

Velocity  (ft/s) 

Target  Position 
Input  Uncertainty  (ft) 

Mean 

Std  Dev 

9 

70.3 

46.3 

772.04 

35.7 

10 

54.0 

30.3 

783.43 

23.7 

12 

76.0 

36.7 

781.22 

28.7 

18 

71.4 

51.8 

783.31 

40.6 

19 

83.8 

49.0 

785.14 

38.5 

22 

64.6 

32.9 

774.41 

25.5 

Mean 

70.0 

41.2 

779.93 

32.1 

-  Note  that  the  standard  deviations  of  the  WSIC-to-SIMLAB  latencies  were  significant  (up 
to  73%  of  mean  value). 
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The  standard  deviation  resulted  in  a  random  variation  in  the  target  position  which  cannot 
be  compensated  for  in  real-time.  Rather,  this  represents  an  uncertainty  in  the  target 
location  input  to  the  SIMLAB  simulation,  analogous  to  measurement  error  in  range  TSPI 
systems. 

The  SIMLAB  simulation  for  the  missile  flyout  is  a  “rate-driven”  simulation.  This  means  that  it 
uses  the  target  velocity  as  an  input  driver  and  integrates  the  velocity  to  determine  the  target 
location  as  an  output.  When  the  target  latitude,  longitude,  and  altitude  computed  by  the 
simulation  were  compared  to  the  target  latitude,  longitude,  and  altitude  input  into  the  simulation, 
significant  differences  were  found. 

-  The  differences  were  largest  in  the  target  latitude  and  were  found  to  increase 
monotonically  with  time,  leading  to  a  “latitude  divergence”  which  was  largest  at  the  end  of 
the  missile  flyout. 

-  This  problem  first  became  evident  during  the  V&V  Mission,  and  the  latitude  divergences 
observed  corresponded  to  disagreements  of  200-300  ft  between  the  WSIC-determined  and 
SIMLAB-computed  positions  at  the  end  of  the  missile  flyout. 

-  The  magnitude  of  the  disagreement  was  reduced  by  increasing  the  target  PDU  update  rate 
from  12.5  Hz  to  20  Hz  for  the  Parametric  Study  Mission.  As  a  result,  the  latitude 
divergences  corresponded  to  disagreements  of  less  than  50  ft  for  this  mission.  An  example 
is  given  in  Figure  4.1. 1.3-1.  (Note  the  “jagged”  features  in  the  solid  curve  of  Fig.  4.1. 1.3- 
1.  These  features  are  like  those  of  curve  (4)  in  Fig.  4.1. 1.2-1  and  correspond  to  the 
uncertainty  in  the  target  latitude  due  to  WSIC-to-SIMLAB  latency  variations). 

-  The  total  divergence  between  the  target  position  input  into  the  SIMLAB  simulation  and 
the  target  position  computed  by  the  SIMLAB  simulation  was  quantified  as  follows: 

—  The  difference  betwe  en  the  target  latitudes  was  computed.  Because  of  the  uncertainty 
in  the  target  latitude  input  into  the  SIMLAB  simulation  (Table  4. 1.1. 3 -3),  the 
difference  also  had  an  uncertainty.  To  minimize  this  uncertainty,  the  last  ten  values  of 
the  difference  in  latitudes  were  averaged,  and  the  standard  deviation  of  these  was  used 
to  estimate  the  uncertainty  in  the  mean  difference. 

-  The  differences  between  the  target  longitudes  and  altitudes  were  also  computed. 
These  varied  randomly  over  the  entire  time  of  the  missile  flyout,  so  that  all  values  of 
these  differences  were  averaged. 

—  The  mean  values  of  the  differences  in  the  latitude,  longitude,  and  altitude  were 
combined  for  a  net  position  divergence  for  each  run  (given  by  the  RSS  of  the 
individual  latitude,  longitude,  and  altitude  differences).  The  uncertainty  in  the  net 
value  was  estimated  by  the  RSS  combination  of  the  standard  deviations.  Results  are  in 
Table  4. 1.1. 3-4.  Note  that  the  mean  net  position  difference  (36.0  ft)  is  comparable  to 
the  mean  position  uncertainty  from  Table  4. 1 . 1 .3-3  (32. 1  ft). 
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Table  4.1.1.3-4.  Difference  in  Target  Position  Computed  by  SIMLAB  Simulation  and 
Target  Position  Input  Into  SIMLAB  Simulation 


Run  # 
(11/19/96) 

Latitude 
Difference  (ft) 

Longitude 
Difference  (ft) 

Altitude 
Difference  (ft) 

Net  Position 
Difference  (ft) 

28.5  ±2.9 

-11.719.3 

6.715.6 

31.51 11.2 

10 

37.212.7 

-5.7111.4 

5.915.7 

38.1113.0 

12 

39.0  ±2.7 

7.0113.3 

-11.015.5 

41.1114.6 

18 

29.7  ±  1.7 

-15.01 17.4 

6.4 1  6.4 

33.9118.6 

19 

15.311.1 

-36.7120.6 

8.2  1  6.4 

40.6121.6 

22 

27.212.1 

-13.51 10.8 

4.815.5 

30.7112.3 

Mean 

29.512.2 

12.6113.8 

3.515.9 

36.0115.2 

The  major  source  of  the  latitude  divergence  appeared  to  be  velocity  integration  errors  in  the 
SIMLAB  simulation.  (Recall  that  the  simulation  integrated  the  target  velocity  to  determine  the 
target  position.) 

-  The  simulation  assumed  that  each  velocity  component  of  the  target  remained  constant 
over  each  integration  period  (the  integration  period  is  the  frame  time  of  the  simulation). 
This  assumption  was  utilized  because  the  integration  must  be  performed  in  real-time, 
without  knowledge  of  how  the  velocity  will  change  during  the  integration  time. 

-  Since  the  target  was  executing  a  constant  velocity  turn,  the  velocity  components  were 
actually  changing  with  time,  so  that  the  SIMLAB  assumption  of  constant  velocity  was  not 
valid.  The  source  of  the  integration  error  is  illustrated  in  Figure  4.1. 1.3-2. 

-  The  magnitude  of  the  integration  error  was  estimated  by  using  the  initial  and  fma  1  target 
velocities  for  each  integration  step  to  compute  the  triangular  integration  error  area 
illustrated  in  Figure  4. 1.1. 3-2,  according  to: 

8v  =  2}(vf-v,)At  (2) 

where  8v  =  velocity  integration  error 

Vf  =  velocity  at  the  end  of  the  integration  time  step 
Vj  =  velocity  at  the  start  of  the  integration  time  step 
At  =  size  of  each  integration  time  step 

E  indicates  the  sum  of  the  terms  over  all  integration  time  steps 

-  Results  of  applying  Equation  2  are  given  in  Table  4.1. 1.3-5.  T  his  table  also  compares  the 
integration  error  estimates  to  the  net  target  position  differences  from  Table  4.1.1 .3-4. 
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Table  4.1.1.3-5.  SIMLAB  Velocity  Integration  Error  Estimates 


Rim# 

Integration  Error 

Net  Position 

(11/19/96) 

Estimate  (ft)  i 

Difference  (ft) 

9 

24.7 

31.5  ±  11.2 

10 

30.0 

38.1  ±13.0 

12 

29.7 

41.1  ±14.6 

18 

35.5 

33.9  ±18.6 

19 

31.4 

40.6  ±21.6 

22 

24.8 

30.7  ±  12.3 

Mean 

29.4 

36.0  ±15.2 

Conclusions  from  Table  4. 1.1. 3-5  are  as  follows: 

The  integration  error  estimate  accounts  for  most,  but  not  all,  of  the  position  divergence. 
—  In  addition  to  assuming  that  the  velocity  was  constant  over  each  integration  step,  it 
was  also  noted  that  frequently  the  target  velocity  components  input  into  the  SIMLAB 
simulation  were  not  being  updated  at  the  PDU  update  rate  set  on  the  NIUs  (as  low  as 
-3  Hz  for  the  velocity  updates  vs.  20  Hz  for  the  PDU  update  rate).  The  target  latitude 
was  computed  by  integration  of  the  north  velocity  component,  and  this  component 
decreased  with  time  during  the  target’s  constant  speed  turn.  Hence,  when  the  velocity 
components  were  not  updated,  the  simulation  used  too  large  of  a  value  for  the  north 
velocity  component,  and  this  aggravated  the  integration  error. 

-  The  total  mean  target  position  error  (36  ft)  is  much  larger  than  the  raw  data-to-PDU 
transform  error  (-1-2  ft),  and  the  uncertainty  in  the  error  is  significant  (-40%  of  the  mean 
value). 

-  The  total  mean  target  position  error  is  also  significantly  larger  than  the  missile  lethal 
radius.  This  means  that  lethality  results  based  on  the  SIMLAB  simulation-computed  miss 
distances  and  on  PDU-computed  miss  distances  can  disagree  on  whether  or  not  the  missile 
killed  the  target.  Because  the  target  representation  in  the  SIMLAB  simulation  does  not 
accurately  represent  the  actual  target  behavior,  the  SIMLAB  value  of  miss  distance  cannot 
be  considered  the  “correct”  value  (even  though  the  missile  behavior  may  be  accurate  for 
the  representation  of  the  target  presented  to  it  in  the  SIMLAB). 

CONCLUSION .  The  target  representation  in  the  SIMLAB  simulation  does  not  accurately  reflect 
the  target  motion  generated  at  the  WSIC.  Unless  this  problem  can  be  solved,  the  LSP 
configuration  cannot  be  used  for  closed-loop  applications  in  which  the  missile  and  the  target 
respond  to  each  other  in  real-time.  A  possible  solution  would  be  improving  the  SIMLAB 
integration  of  target  velocity  to  determine  position  by  using  the  target  acceleration  at  the  start  of 
each  integration  step  to  partially  correct  for  the  change  in  velocity  during  the  step.  Also,  the 
target  velocity  component  values  input  into  the  SIMLAB  simulation  need  to  be  updated  at  the 
same  rate  as  the  other  entity  state  data. 
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Figure  4.1.1.3-1.  Latitude  Divergence  of  Target  Trajectory  in  Run  #12  (11/19/96) 


Verification  of  AIM-9  Flvout  Data  .  The  missile  entity  state  data  were  checked  according  to  the 
four-point  comparison  illustrated  in  Figure  4. 1.1. 1-1.  The  latitude,  longitude,  and  altitude 
generated  at  the  end  of  the  missile  flyout  were  used  to  eliminate  dead  reckoning  adjustments  to 
the  PDU  values,  and  results  are  given  in  Table  4. 1.1. 3-6  (latitude  and  longitude  differences  were 
directly  measured  and  converted  into  distances  in  the  table).  As  before,  the  actual  PDU  data  were 
converted  into  latitude,  longitude,  and  altitude  for  the  comparison. 


Table  4.1.1.3-6.  Verification  of  Missile  Positional  Data  Passed  From  SIMLAB  to  WSSF 

(from  Run  #9  on  11/19/96) 


Location 

Latitude 

Longitude 

Altitude 

Delta  Lat 
(deg) 

Delta  Lat 

(ft) 

Delta 
Lon  (deg) 

Delta 
Lon  (ft) 

Delta  Alt 

SIMLAB  raw 

35.76203053 

-117.6784207 

10292.3 

SIMLAB  logger 

35.76203378 

-117.6784249 

10291 

3.25E-06 

1.188 

-4.2E-06 

-1.245 

-1.3 

WSSF  logger 

35.76203378 

-117.6784249 

10291 

0 

0 

0 

0 

0 

WSSF  raw 

35.76203397 

1.9E-07 

0.069 

3E-06 

0.889 

1.3 

Net  of  Deltas 

3.44E-06 

1.257 

-1.2E-06 

-0.356 

0 

Total  of  Nets 

1.31 

The  conclusions  are: 

-  The  positions  from  the  PDUs  agreed  with  the  SIMLAB  raw  data  to  within  2  .2  ft  (RSS  of 
deltas  from  second  row  of  Table  4. 1.1. 3-6).  This  reflected  the  accuracy  of  the  TCAC 
algorithms  for  converting  the  PDU  coordinates  into  latitude,  longitude,  altitude,  as  well  as 
the  accuracy  of  the  lab  frame-to-PDU  frame  conversion. 

-  The  PDU  data  were  not  changed  during  transmission  from  the  SIMLAB  to  the  WSSF. 

-  The  net  accuracy  for  transmitting  target  positional  data  from  the  SIMLAB  to  the  WSSF 
(via  PDUs)  is  about  1.3  ft  (the  total  was  obtained  from  the  RSS  of  the  individual  nets), 
and  is  acceptable  for  the  LSP  scenario.  Note  that  the  comparison  of  raw  simulation  data 
were  direct  and  did  not  require  coordinate  transformations. 

However,  a  problem  was  found  with  the  missile  flyout  when  the  SIMLAB  inertial  reference  frame 
data  was  compared  to  latitude,  longitude,  and  altitude: 

-  The  SIMLAB  simulation  transformed  the  incoming  latitude,  longitude,  and  altitude  of  the 
shooter  and  target  into  its  own  reference  frame: 

--  The  horizontal  component  of  the  target  velocity  vector  (referred  to  as  targe  t  bearing) 
defined  an  x-axis  in  the  horizontal  plane.  The  origin  was  the  x-component  of  the 
launch  location. 

-  The  z-axis  was  also  in  the  horizontal  plane  and  perpendicular  to  the  x-axis.  The  origin 
was  the  z-component  of  the  launch  location. 

-  The  y-axis  was  in  the  local  vertical  direction  and  the  y-component  was  identical  to  the 
entity  altitude. 

-  The  missile  flyout  was  computed  in  the  simulation  in  terms  of  this  x,y,z  reference  frame. 
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When  ranges  between  the  missile  and  the  target  were  computed  using  the  x,y,z 
coordinates  of  each  and  compared  to  ranges  computed  using  the  latitude,  longitude,  and 
altitude  of  each,  the  initial  (launch)  range  was  found  to  agree  very  well,  but  the  final  range 
was  found  to  disagree  by  ~1000  ft. 

~  The  disagreement  is  illustrated  in  Figure  4. 1.1. 3-3  which  shows  the  “God’s-eye”  view 
of  the  missile  and  target  trajectories  observed  in  the  SIMLAB  reference  frame  (Fig. 
4.1.1.3-3(a))  compared  to  the  trajectories  as  determined  by  PDU  data  (Fig.  4.1. 1.3- 
3(b)). 

—  For  the  comparison,  the  SIMLAB  trajectory  data  were  rotated  by  an  angle  equal  to 
the  target  bearing  at  launch;  this  rotates  the  SIMLAB  x-axis  into  the  north 
direction. 

—  Note  that  the  missile  trajectories  agree,  but  not  the  target  trajectories.  As  a  result, 
the  missile  guides  to  the  target  according  to  SIMLAB  data,  but  does  not  according 
to  PDU  data. 

The  problem  of  the  range  disagreement  was  traced  to  an  error  in  initializing  the  target  location  in 
the  x,y,z  reference  frame. 

-  Instead  of  computing  the  initial  t  arget  location  in  the  x,z  plane  by  using  the  target  bearing, 
this  was  done  by  incorrectly  using  the  target  heading. 

-  In  the  LSP  scenarios,  the  angle  between  the  target  bearing  and  heading  was  typically  5.5  °. 

This  angular  difference  over  the  launch  range  resulted  in  an  offset  in  the  initial  x,y,z 
position  of  the  target  of  ~1000  ft. 

-  This  initial  offset  applied  to  each  SIMLAB  simulation  frame,  so  that  the  missile  was  flying 
toward  a  target  in  the  SIMLAB  x,y,z  reference  frame  which  was  always  offset  from  the 
true  target  position  by  -1000  ft.  The  range  can  be  corrected  by  applying  the  offset  to  each 
simulation  time  step. 

—  This  constant  offset  value  applied  to  each  simulation  frame  because  the  simulation 
used  the  initial  location  of  the  target  at  launch  and  then  determined  the  location  change 
in  each  frame  by  integrating  the  target  velocity. 

-  Figure  4. 1 . 1 .3-4  shows  that  by  first  translating  the  SIMLAB  target  data  and  then  rotating 
the  data  values,  the  target  trajectory  from  SIMLAB  data  overlays  the  trajectory  from  PDU 
data. 

-  Note  that  the  target  trajectories  do  not  exactly  overlay,  because  of  the  latitude 
divergence  problem. 
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/  -  Trajectories  as  observed  in  the  SIMLAB  reference  frame 
are  rotated  from  that  frame  by  an  angle  equal  to  the 
target  bearing  at  launch 


-  Missile  trajectory  overlays  trajectory  from  missile  PDUs 

-  Missile  guides  to  target,  as  in  SIMLAB  reference  frame 


East  Distance 


Figure  4.1.1.3-3(a).  Missile  and  Target  Trajectories  from  SIMLAB  Data  (Run  #12  on 

11/19/96) 


Figure  4.1.1.3-3(b).  Missile  and  Target  Trajectories  from  PDU  data  (Run  #12  on  11/19/96) 
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Figure  4.1.1.3-4.  Illustration  of  Correction  to  Align  Target  Trajectories  From  SIMLAB  and  PDU  Data  (Run  #12  on  11/19/96) 


4.1.1.4  Verification  Summary 

The  results  for  the  verification  objective  are  summarized  as  follows: 

The  error  in  transforming  from  the  raw  simulation  positional  data  to  entity  state  PDU  data 
is  1-2  ft. 

—  This  is  an  acceptable  value. 

-  There  were  no  errors  in  transforming  velocity  and  orientation  data. 

No  errors  were  found  in  transmitted  PDU  data  values.  All  PDU  data  passed  between 
nodes  matched. 

-  There  were  a  number  of  errors  in  the  target  positidnal  data,  as  represented  in  the  SIMLAB 
simulation. 

—  Random  latency  variations  introduced  uncertainty  in  the  targe  t  position.  The  random 
nature  of  these  variations  prevents  implementing  a  deterministic  real-time  correction 
for  all  latency  effects. 

-  The  target  latitude  determined  by  the  SIMLAB  simulation  diverged  from  the  WSIC 
value  during  the  missile  flyout.  The  cause  appears  to  be  primarily  due  to  the  simplistic 
method  used  by  the  SIMLAB  to  determine  target  position  by  integrating  its  velocity. 

A  more  sophisticated  integration  technique  and  higher  velocity  update  rate  could 
significantly  reduce  the  divergence. 

-  The  target  representation  in  the  SIMLAB  simulation  coordinate  frame  was  wrong  due 
to  an  error  in  the  coordinate  frame  transformation  used.  This  was  fixed  after  the 
Parametric  Study  Mission. 

4.1.2  Validation 

The  objective  of  the  validation  process  is  to  demonstrate  that  the  performance  of  the  AIM-9M- 
8/9  in  the  SIMLAB  facility,  driven  by  the  linked  simulators  (the  WSSF  as  the  shooter  and  the 
WSIC  as  the  target),  was  consistent  with  that  of  an  actual  AIM-9M-8/9  missile  live  fire  profile 
under  the  same  conditions. 

4.1.2.1  Validation  Test  Method 

Data  for  validation  were  collected  during  the  Parametric  Study  Mission  (11/19/96)  runs  which 
replicated  the  live  test  profile  and  during  various  SIMLAB  standalone  runs. 

The  AIM-9M-8/9  JIOT&E  live  fire  test  selected  as  the  baseline,  LPN-15,  is  depicted  in  Figure 
4. 1.2. 1-1.  The  pilots  in  the  WSSF  and  WSIC  flight  simulators  attempted  to  replicate  the  live  fire 
profile  by  using  scripted  passes.  The  replication  profile  (with  flare  countermeasures)  was 
designated  as  Profile  V2  and  was  baselined  on  the  actual  shooter  and  target  parameters  and  event 
timing  in  die  LPN-15  scenario.  The  V&V  approach  involved  comparing  the  performance  of  the 
simulated  missile  (especially  its  flyout)  to  that  of  the  real  missile  to  establish  the  validity  of  the 
ADS  configuration.  » 
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AIM-9M- 


F/A-18C 

11,300  ft/  0.71  mach 
0  °  angle  off  boresight 


OF-86 

•  10,400  ft  /  0.72  mach 

•  58  0  angle  off  tail 

•  3.6  g  level  turn 

•  flare  countermeasures 


Figure  4.1.2.1-1.  AIM-9M-8/9  Live  Fire  Profile  (LPN-15, 9  June  93) 


The  V2  replication  profiles  were  executed  in  two  basic  configurations  as  follows: 

-  Manual  Linked  Runs  .  The  shooter  and  target  simulators  were  each  flown  and  controlled 
by  pilots.  Each  aircraft  simulator  was  started  manually  by  local  operators  when  directed 
by  the  test  controller  at  the  TCAC.  Flares  were  released  manually.  Each  simulator  was 
initialized  with  pre-established  sets  of  initial  conditions  that  were  refined  during  dry  runs 
so  that  the  two  aircraft  were  able  to  reliably  replicate  the  LPN-15  launch  conditions. 

.  SIMLAB  Standalone  Puns  .  These  runs  were  performed  without  linking  the  SIMLAB  to 
the  other  simulators.  They  were  executed  by  preprogramming  the  SIMLAB  with  the 
following  profiles  (including  all  timed  events): 

—  The  baseline  LPN-15  profile.  These  runs  are  used  to  verify  SIMLAB  operation  and  to 
provide  a  baseline  for  missile  flyout  results  from  the  linked  configuration. 

-  Launch  conditions  determined  by  Monte  Carlo  sampling  of  the  shooter  and  target 
inputs  about  the  LPN-15  profile.  The  range  of  conditions  which  were  sampled  was 
the  LPN-15  shot  box  (Table  4.1. 1.2-1).  These  runs  are  used  to  develop  criteria  for 
initial  (i.e.,  launch)  conditions  and  for  the  validity  of  missile  flyout  results  from  the 
linked  runs. 

-  The  manual  linked  run  which  best  replicated  LPN-15.  These  runs  were  used  to 
determine  the  run-to-run  variations  in  the  SIMLAB  output  when  the  shooter  and 
target  inputs  were  held  constant  (for  “best”  manual  run  conditions)  and  when  the  ADS 
network  was  not  used 
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4.1. 2.2  Validation  Analysis  Method 

The  SIMLAB  was  first  run  in  the  standalone  mode  using  the  LPN-15  five  fire  test  launch 
conditions  and  target  trajectoiy  as  inputs.  The  purpose  was  to  baseline  the  SIMLAB  missile 
flyout  against  the  live  test  data.  The  SIMLAB  was  run  20  times  using  the  same  LPN-15  inputs 
each  time,  because  the  seeker  performance  varies  slightly  run-to-run.  The  envelope  of  missile 
flyouts  from  the  SIMLAB  runs  was  then  compared  to  the  LPN-15  data  using  the  quantitative 
evaluation  described  below. 

Assuming  reasonable  agreement  between  the  SIMLAB  standalone  results  and  the  LPN-15  data, 
the  data  recorded  at  the  SIMLAB  for  each  run  of  the  validation  profile  were  to  be  compared  to 
results  from  the  SIMLAB  standalone  runs.  The  data  used  in  the  comparison  were  primarily  the 
trajectories  of  the  missile  and  target.  The  criteria  for  determining  data  matches  from  the 
comparisons  were  to  be  derived  from  SIMLAB  standalone  Monte  Carlo  runs  of  the  validation 
profile  (V2),  as  follows: 

-  The  expected  range  of  launch  conditions  was  given  by  one  of  the  following: 

-  The  exact  conditions  of  LPN- 1 5  (as  in  the  runs  for  comparison  with  LPN- 1 5). 

—  The  shot  box  about  the  LPN-15  conditions. 

-  The  exact  conditions  of  the  manual  linked  run  which  best  replicated  LPN-15. 

-  The  launch  conditions  and  target  trajectories  were  sampled  from  uniform  distributions 
consistent  with  the  ranges  of  the  values. 

-  SIMLAB  standalone  results  for  missile  trajectories  were  examined  to  determine  the 
bounding  values  (see  Fig.  4.1. 2.2-1). 

-  The  range  of  SIMLAB  standalone  results  was  reviewed  to  determine  if  the  range 
represented  reasonable  comparison  criteria. 

The  data  recorded  in  the  SIMLAB  from  the  linked  missions  were  compared  to  the  data  from  one 
of  the  three  types  of  SIMLAB  standalone  runs.  The  comparison  of  the  missile  results  was  either 
quantitative  (if  all  SIMLAB  inputs  were  within  acceptable  ranges)  or  qualitative  (if  any  SIMLAB 
input  was  outside  acceptable  ranges).  The  determination  of  the  acceptability  of  SIMLAB  inputs 
was  as  follows: 

Initial  launch  conditions 

-  Each  parameter  recorded  in  the  SIMLAB  was  compared  to  the  shot  box. 

—  If  all  initial  launch  parameters  were  wit  hin  the  shot  box,  the  SIMLAB  results  were 
quantitatively  evaluated  as  described  below. 

—  If  any  initial  launch  parameter  was  outside  the  shot  box,  the  SIMLAB  results  were 
qualitatively  evaluated  as  described  below. 

-  Target  velocity  profile  and  trajectory 

--  The  target  velocity  and  altitude  during  the  missile  flyout  were  compared  to  the  shot 
box. 

—  If  both  target  parameters  were  within  the  shot  box,  the  SIMLAB  results  were 
quantitatively  evaluated  as  described  below. 
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—  If  either  target  parameter  was  outs  ide  the  shot  box,  the  SIMLAB  results  were 
qualitatively  evaluated  as  described  below. 

-  Target  IR  intensity 

-  The  discrete  value  for  the  apparent  IR  intensity  of  the  target  (relative  to  the  flare 
intensity  at  the  time  of  first  flare  deployment)  was  preset  in  the  SIMLAB  (by  a 
combination  of  IR  aperture  size  and  blackbody  temperature).  This  intensity  was 
checked  before  the  linked  missions  to  verify  that  the  value  measured  in  the  SIMLAB 
was  within  the  acceptable  range,  as  determined  by  the  AIM-9  expert. 

-  Countermeasure  parameters 

--  The  discrete  value  for  the  flare  dispense  time  was  compared  to  the  acceptable  range  of 
values  determined  by  the  AIM-9  expert  and  was  judged  to  be  either  acceptable  (it  was 
within  the  acceptable  range)  or  unacceptable  (it  was  outside  the  acceptable  range). 

—  The  other  flare  parameters  (dispense  rate,  temperature,  ejection  angle)  were  all  preset 
in  the  SIMLAB.  The  value  of  each  of  these  parameters  was  checked  before  the  linked 
missions  to  verify  that  the  correct  value  was  selected. 

The  quantitative  evaluation  of  SIMLAB  output  (used  when  all  SIMLAB  inputs  were  acceptable) 
was  to  be  as  follows: 

-  Missile  trajectory 

—  SIMLAB  standalone  results  for  missile  trajectories  were  examined  to  determine  the 
bounding  values,  as  described  above.  The  steps  for  doing  this  with  the  trajectory  data 
are  shown  in  Figure  4.1. 2.2-1  as  (a)  and  (b). 

—  The  plot  of  each  parameter  recorded  in  the  SIMLAB  was  compared  to  the  acceptable 
range  of  plots  determined  from  the  Monte  Carlo  runs  by  the  AIM-9  expert. 

—  Each  parameter  was  judged  to  be  either  acceptable  (all  points  on  plot  He  within  the 
acceptable  range:  see  Fig.  4.1.2.2-l(c))  or  unacceptable  (some  points  on  plot  lie 
outside  the  acceptable  range,  i.e.,  those  in  interval  b  in  Fig.  4.1.2.2-l(d)). 

-  Missile  seeker  TM  signals 

-  These  were  analyzed  by  the  AIM-9  expert  based  on  their  familiarity  with  five  fire 
results  from  many  tests.  The  expert  determined  if  any  TM  signals  appeared  to  be 
invalid  and  what  characteristics  made  them  appear  to  be  invalid. 

Missile  terminal  conditions 

-  Each  discrete  parameter  recorded  in  the  SIMLAB  was  compared  to  the  acceptable 
range  of  values  determined  from  the  Monte  Carlo  runs  by  the  AIM-9  expert. 
Parameters  to  be  used  included: 

—  Miss  distance 

—  Terminal  range  (range  when  SIMLAB  flyout  simulation  stops  at  0.1  sec  before 
intercept) 

—  Final  range  rate  (range  rate  when  SIMLAB  flyout  simulation  stops  at  0.1  sec 
before  intercept) 

~  Each  discrete  parameter  was  judged  to  be  either  acceptable  (it  was  within  the 
acceptable  range)  or  unacceptable  (it  was  outside  the  acceptable  range). 
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(a)  Compilation  of  SIMLAB 
Monte  Carlo  Runs 


(b)  Envelope  of  SIMLAB 
Monte  Carlo  Runs 


(c)  Comparison  of  Single  Trial  to 
Envelope  with  All  Points  Valid 


(d)  Comparison  of  Single  Trial  to 
Envelope  with  Interval  a  Data 
Valid  and  Interval  b  Data  Invalid 


Figure  4.1. 2.2-1.  Steps  in  Quantifying  the  Validity  of  Plotted  Data 


The  qualitative  evaluation  of  SIMLAB  output  was  used  when  any  SIMLAB  input  was 
unacceptable.  This  involved  comparison  of  the  general  trends  and  shapes  of  missile  trajectory 
plots  and  TM  data  by  the  AIM-9  expert.  This  evaluation  will  be  based  on  the  expert’s  familiarity 
with  live  fire  results  from  many  tests  and  the  degree  to  which  the  launch  conditions,  target 
trajectory  and  velocity,  or  flare  dispense  time  varies  from  the  acceptable  range. 

Unacceptable  matches  were  further  analyzed  to  determine  the  cause  of  the  mismatch  and  the 
whether  the  entire  trial  was  to  be  judged  valid  or  invalid: 

-  If  any  SIMLAB  input  was  unacceptable,  the  values  at  the  SIMLAB  high-fidelity  simulator 
input  and  at  the  originating  high-fidelity  simulation  computer  output  were  compared. 

-  If  the  originating  value  agreed  with  the  SIMLAB  input  (i.e.,  both  were  unacceptable), 
the  cause  of  the  mismatch  was  judged  to  be  non-ADS-related. 

-  If  the  originating  value  was  acceptable,  the  cause  of  the  mismatch  was  judged  to  be 
ADS-related  and  was  further  analyzed  under  Test  Objective  4. 

-  The  qualitative  evaluation  of  the  SIMLAB  output  by  the  AIM-9  expert  determined  if 
the  trial  was  valid  or  invalid. 

-  If  all  SIMLAB  inputs  were  acceptable,  but  some  of  the  SIMLAB  outputs  were 
unacceptable,  the  cause  was  assumed  to  be  related  to  operation  of  the  SIMLAB  and  was 
non-ADS-related.  The  overall  performance  of  the  missile  simulation  was  assessed  by  the 
AIM-9  experts  to  determine  if  the  trial  was  valid  or  invalid. 
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4.1. 2.3  Validation  Results 


The  following  missile  simulation  parameters  were  checked  by  the  AIM-9  experts  before  linked 
testing  began  and  were  determined  to  be  acceptable  (these  were  not  varied  throughout  the  linked 
testing  periods): 

The  apparent  IR  intensity  of  the  target. 

-  The  preset  flare  parameters  (dispense  rate,  temperature,  ejection  angle). 

The  acceptability  of  SIMLAB  results  for  quantitative  evaluation  was  subsequently  based  on: 

-  All  launch  parameters  being  within  the  shot  box. 

-  The  target  executing  a  flat  (constant  altitude),  constant  acceleration  turn  during  the  missile 
flyout. 

SIMLAB  Standalone  Runs  of  LPN-15.  Exact  Launch  Conditions 

The  SIMLAB  standalone  runs  were  performed  which  used  the  exact  LPN-15  launch  conditions. 
The  results  of  the  missile  trajectories  from  these  runs  were  used  to  determine  the  envelopes  of 
bounding  values  (as  described  in  Fig.  4.1. 2.2-1).  The  envelopes  were  then  compared  to  the  actual 
LPN-15  data  in  Figures  4.1.2.3-1  through  4.1.2.3-3.  These  figures  show  the  following: 

-  There  were  variations  in  the  missile  flyouts  from  run-to-run,  even  when  identical  launch 
conditions  and  target  trajectories  were  input  to  the  SIMLAB  on  each  run.  The  AIM-9M 
seeker  did  not  respond  to  the  inputs  in  exactly  the  same  way  every  time.  However,  the 
run-to-run  variations  are  relatively  small  resulting  in  narrow  envelopes  during  the 
midcourse  phase  of  the  flyout. 

-  The  “God’s-eye”  view  (Fig.  4.1.2.3-1)  shows  very  good  agreement  between  the  SIMLAB 
results  and  the  live  test. 

-  The  side  views  (Figs.  4.1.2.3-2  and  4.1.2.3-3)  show  less  agreement.  For  a  short  time  at 
the  start  of  its  flyout,  the  SIMLAB  missile  flew  slightly  above  the  live  missile.  However, 
for  most  of  its  flyout,  the  SIMLAB  missile  flew  significantly  below  the  live  missile  and 
approached  the  target  from  a  smaller  angle  relative  to  the  horizontal  direction. 

—  The  live  missile  appeared  to  take  longer  for  i  ts  initial  guidance  correction  (at  the  start 
of  its  flyout,  the  AIM-9  flies  without  any  steering  in  order  to  safely  separate  from  the 
launch  aircraft).  This  resulted  in  the  live  missile  being  lower  than  the  simulated  missile 
and  having  to  compensate  by  flying  a  flatter  trajectory  to  the  target. 

~  These  differences  apparently  resulted  in  the  simulated  missile  having  slightly  longer 
(~3%  longer)  times-of-flight  (TOF)  and  much  smaller  miss  distances  (~20%  of  LPN- 
15  value). 

-  In  spite  of  these  differences,  t  he  simulated  missile  responded  to  the  target  and  correctly 
guided  to  it.  When  the  AIM-9  expert  examined  the  trajectory  plots,  along  with  the  seeker 
telemetry  data,  his  judgment  was  that  the  performance  of  the  simulated  missile  was  valid 
for  the  given  scenario.  Note  the  qualitative  features  in  both  the  live  and  simulated  missile 
flyouts: 
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—  An  initial  straight  “safe-separation”  segment. 

-  A  distinct  guidance  correction  at  the  end  of  the  “safe-separation”  segment. 

--  Continual  and  smooth  closing  on  the  taiget  with  no  gain  in  missile  altitude. 

-  Note  that  the  simulated  target  was  modeled  to  execute  a  perfectly  flat  right  turn,  whereas 
the  live  target’s  altitude  varied  slightly.  The  differences  in  the  target  trajectory  were  minor 
and  did  not  significantly  affect  the  missile  flyout. 

CONCLUSIONS 

-  The  SIMLAB  standalone  simulation  of  the  missile  flyout  for  the  LPN-15  conditions  was 
valid.  (Note  that  the  AIM-9  Program  Office  has  accredited  the  SIMLAB  as  a  valid 
simulation  in  support  of  AIM-9  testing.) 

-  There  were  some  minor  differences  in  the  simulated  missile  flyout  compared  to  LPN-15 
which  reflect  simulation  fidelity  and/or  validity  of  the  LPN-15  data. 

The  LPN-15  data  do  not  necessarily  give  a  more  accurate  representation  of  the  missile  in 
this  scenario. 

—  The  LPN-15  data  were  derived  from  range  measurements  and  are  subject  to 
inaccuracies  and  uncertainties. 

—  The  LPN-15  data  represent  only  a  single  realization  of  the  missile  behavior  for  this 
scenario.  As  the  SIMLAB  results  show,  there  were  run-to-run  variations  in  the 
performance  of  the  missile  seeker  and  guidance  for  the  exact  same  scenario. 

-  For  the  above  reasons,  the  envelope  of  SIMLAB  standalone  results  were  judged  to  give  a 
better  standard  for  comparison  with  the  linked  results  than  did  the  LPN-15  data.  This 
comparison  was  used  to  determine  if  linking  the  simulators  resulted  in  any  degradation  in 
the  SIMLAB  simulation  performance  (the  SIMLAB  in  the  linked  configuration  cannot 
represent  the  missile  behavior  any' better  than  in  the  standalone  configuration). 

-  The  features  noted  in  both  the  live  and  the  standalone  simulated  missile  flyouts  can  be 
used  for  qualitative  validation  of  the  linked  missile  flyouts. 

-  The  missile  TOF  gave  a  better  metric  for  quantifying  overall  missile  behavior  than  did  the 
miss  distance. 

~  The  SIMLAB  was  not  designed  to  accurately  model  the  terminal  engagement.  The 
missile  flyout  simulation  stops  at  0.1  sec  from  intercept,  and  the  miss  distance  is 
estimated  by  dead  reckoning  the  target  and  missile  velocity  vectors  to  the  point  of 
closest  approach  (the  last  missile  entity  state  PDU  is  generated  when  the  missile  flyout 
simulation  stops). 

~  The  distribution  of  miss  distances  from  the  SIMLAB  results  did  not  give  additional 
insight  into  the  validity  of  the  overall  missile  performance. 

-  Consequently,  the  miss  distance  was  not  used  in  subsequent  validity  analyses. 
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"  Standalone  Runs  -  Missile  Envelope 
-Standalone  Runs  -  Target 
“LPN-15- Missile 


Standalone  Runs  -  Missile  Envelope 
Standalone  Runs  -  Target 


Figure  4.1.2.3-2.  Envelope  of  SIMLAB  Standalone  Runs  (using  exact  LPN-15  launch  conditions)  Compared  to  LPN-15  Data  - 

Side  View  #1 


Standalone  Runs  -  Missile  Envelope 
Standalone  Runs  -  Target 
LPN-1 5  -Missile 
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Figure  4.1.2.3-3.  Envelope  of  SIMLAB  Standalone  Runs  (using  exact  LPN-15  launch  conditions)  Compared  to  LPN-15  Data  - 

Side  View  #2 


SIM  L  AB  Standalone  Runs  of  LPN-15.  Monte  Carlo  Launch  Conditions 


The  SIMLAB  standalone  runs  were  performed  in  which  the  shooter  and  target  launch  conditions 
were  determined  by  Monte  Carlo  sampling  of  the  shooter  and  target  inputs  about  the  LPN-15 
profile.  The  launch  conditions  for  each  run  were  randomly  selected  and  could  have  any  value 
within  the  shot  box  with  equal  probability.  The  results  of  the  missile  trajectories  from  these  runs 
were  used  to  determine  the  envelopes  of  bounding  values  (as  described  in  Fig.  4.1. 2.2-1).  The 
envelopes  were  then  to  be  compared  to  the  missile  flyouts  from  the  linked  runs.  The  resulting 
envelopes  are  shown  in  Figures  4. 1.2. 3-4  through  4. 1.2. 3-6.  These  figures  show  the  following: 

-  The  envelope  for  the  “God’s-eye,”  or  horizont  al,  view  (Fig.  4.1. 2.3-4)  shows  the  missile 
flyouts  all  originating  from  a  common  point.  This  is  because  the  SIMLAB  reference  frame 
was  defined  such  that  the  missile  launch  point  was  always  the  origins  of  the  down  range 
and  cross  range  axes.  This  envelope  broadened  as  the  missile  flew  out  toward  the  target 
because  of  variations  in  the  lead  and  target  aspect  angles  selected  for  the  various  runs. 
The  resulting  funnel  was  over  5000  ft  wide  at  the  end  of  the  missile  flyouts.  This  funnel 
might  be  able  to  identify  invalid  flyouts  in  which  the  discrepancy  occurred  near  the  launch 
point  (i.e.,  shortly  after  launch).  However,  significant  deviations  from  a  correct  missile 
trajectory  might  occur  later  in  the  flyout  and  not  fall  outside  this  envelope. 

-  The  envelopes  for  the  side  views  (Figs.  4.1. 2.3-5  and  4.1. 2.3-6)  neither  originate  nor 
terminate  at  a  common  point.  This  is  because  the  SIMLAB  reference  frame  used  the 
actual  missile  and  target  altitudes  in  the  vertical  direction.  The  resulting  envelopes  were 
nearly  constant  in  size  in  the  vertical  direction  throughout  the  missile  flyouts.  This  size 
was  about  the  maximum  allowable  range  of  altitudes  for  either  the  shooter  or  the  target  at 
launch,  1000  ft.  As  with  the  horizontal  envelope,  significant  deviations  from  a  correct 
missile  flyout  could  be  “hidden”  inside  such  a  broad  envelope. 

These  envelopes  were  too  broad  for  distinguishing  between  acceptable  and  unacceptable  missile 
trajectories  from  the  linked  runs.  This  was  demonstrated  to  be  the  case  from  the  following: 

-  When  the  missile  flyouts  from  the  Mission  Rehearsal  and  the  V&V  Mission  were 
qualitatively  examined,  the  missile  was  seen  to  be  improperly  lofting  to  an  altitude  above 
ftie  shooter  before  finally  losing  altitude  to  intercept  the  target.  This  was  improper  missile 
behavior,  since  the  shooter  was  at  a  higher  altitude  than  the  target.  The  LPN-15  results 
(Figs.  4.1. 2.3-2  and  4.1. 2.3-3)  show  that  the  missile  should  not  loft  under  these 
conditions. 

However,  when  the  invalid  lofting  trajectories  wer  e  compared  to  the  vertical  envelopes,  a 
number  of  these  invalid  flyouts  fell  entirely  within  the  envelopes  and  would  have  been 
judged  valid  according  to  the  envelope  criteria.  An  example  from  the  V&V  Mission 
(10/29/96)  is  shown  in  Figure  4. 1.2. 3-7. 

Hence,  the  validation  technique  of  comparing  the  missile  flyouts  from  the  linked  runs  to  these 
envelopes  (those  resulting  from  the  launch  conditions  being  randomly  selected  from  the  shot  box) 
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failed.  The  envelopes  could  be  narrowed  by  significantly  reducing  the  shot  box,  but  the  pilots 
would  not  have  been  able  to  consistently  achieve  a  much  narrower  box. 

CONCLUSIONS 


The  quantitative  validation  technique  based  on  comparing  the  missile  flyouts  from  the 
linked  runs  to  the  envelopes  resulting  from  Monte  Carlo  sampling  of  launch  conditions 
had  to  be  abandoned.  It  was  incapable  of  identifying  many  of  the  invalid  lofting  missile 
trajectories. 

The  missile  trajectories  from  the  SEMLAB  standalone  runs  all  have  similar  shapes  and  the 
same  qualitative  features: 

-  An  initial  straight  “safe-separation”  segment. 

-  A  distinct  guidance  correction  at  the  end  of  the  “safe-separation”  segment. 

-  Continual  and  smooth  closing  on  the  target  with  no  gain  in  missile  altitude. 

-  The  envelopes  resulting  from  SIMLAB  standalone  runs  which  all  used  the  same  launch 
conditions  and  target  trajectory  were  narrow  enough  to  be  useful  for  comparisons  with 
linked  results  (see  Figs.  4.1. 2.3-1  through  4. 1.2. 3-3). 

-  The  validation  method  to  be  used  for  the  linked  runs  had  to  be  modified  to  util  ize  both 
qualitative  and  quantitative  validation. 

The  qualitative  method  was  to  compare  the  shape  of  the  missile  trajectory  to  that  for 
LPN-15  by  looking  for  all  the  qualitative  features  noted  above. 

-  This  method  identified  the  missile  lofting  as  an  invalid  flyout. 

-  The  quantitative  method  was  to  involve  SIMLAB  standalone  runs  using  the  linked 
result  in  which  the  launch  conditions  most  closely  matched  those  of  LPN-15. 

—  On  each  trial,  the  exact  launch  conditions  of  the  best  linked  run  were  to  be  ised. 

—  The  results  of  20  missile  flyouts  were  to  be  used  to  define  an  envelope  of  valid 
flyouts. 

-  The  flyout  of  the  linked  run  was  to  be  validated  by  comparing  it  to  the  envelope. 
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Standalone  Runs  -  Missile  Envelope 
Standalone  Runs  -  Target  Envelope 


Cross  Range  Distance 

Figure  4.1.2.3-4.  Envelope  of  SIMLAB  Standalone  Runs  (using  launch  conditions  randomly  selected  from  shot  box)  - 

“God’s-Eye”  View 


Standalone  Runs  -  Missile  Envelope 
Standalone  Runs  -  Target  Envelope 


Envelope  of  SIMLAB  Standalone  Runs  (using  launch  conditions  randomly  selected  from  shot  box) 

Side  View  #1 
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Down  Range  Distance 

Figure  4.1.2.3-6.  Envelope  of  SIMLAB  Standalone  Runs  (using  launch  conditions  randomly  selected  from  shot  box)  - 

Side  View  #2 


-■—Run  #25  -  Missile 
r  Run  #25  -  Target 

Standalone  Runs  -  Missile  Envelope 


Figure  4.1.2.3-7.  Missile  Flyout  for  Run  #25  (10/29/96)  Compared  to  Envelope  of  SIMLAB  Standalone  Runs  (using  launch 

conditions  randomly  selected  from  shot  box)  -  Side  View  #1 


Validation  of  Linked  Results 

Qualitative  verification  of  the  Mission  Rehearsal  and  V&V  Mission  results  revealed  that  the 
missile  was  improperly  lofting  to  an  altitude  above  the  shooter  before  finally  losing  altitude  to 
intercept  the  target.  The  lofting  was  subsequently  determined  to  be  due  to  incorrectly  inputting 
the  down  component  of  shooter  velocity  into  the  SIMLAB  simulation,  as  follows: 

-  The  shooter  was  at  a  higher  altitude  than  the  target.  The  shooter  boresighted  on  the 
target  prior  to  launch  by  dropping  its  nose.  This  caused  the  shooter  to  always  be  losing 
altitude  at  launch. 

-  Since  the  shooter  was  losing  altitude,  it  had  a  positive  component  of  down  velocity  (the 
vertical  velocity  component  used  by  the  NAWCWPNS  simulations). 

-  Positive  down  velocities  were  incorrectly  interpreted  by  the  simulation  as  the  shooter  (and 
missile)  gaining  altitude  at  launch.  (  The  down  velocities  “signs”  were  reversed  in  the  SSC 
code  in  the  SIMLAB  and  the  WSSF  HP/NIU  computer  code.) 

-  As  a  result,  the  missile  thought  it  was  initially  going  up  at  launch  when  it  really  should 
have  been  going  down. 

The  lofting  problem  was  fixed  for  the  Parametric  Study  Mission  (1 1/19/96),  as  was  verified  using 
the  qualitative  verification  method.  Figures  4.1. 2.3-8  through  4.1.2.3-13  compare  the  results  of 
the  Parametric  Study  Mission  V2  runs  with  the  envelopes  of  SIMLAB  standalone  runs  which 
used  the  exact  LPN-15  launch  conditions.  These  figures  show  the  following  correct  qualitative 
features: 

-  The  missile  lofting  problem  had  been  solved. 

-  An  initial  straight  “safe-separation”  segment. 

-  A  distinct  guidance  correction  at  the  end  of  the  “safe-separation”  segment. 

-  The  missile  smoothly  guided  on  the  target  representation  it  was  given. 

The  six  Parametric  Study  Mission  V2  runs  were  examined  to  determine  which  run  best  matched 
LPN-15  for  the  quantitative  validation.  The  following  factors  were  considered: 

-  The  smallest  absolute  differences  between  LPN-15  and  the  linked  run  for  values  of  target 
aspect  angle,  lead  angle,  flare  deployment  time,  and  missile  TOF. 

-  The  smallest  altitude  variation  of  the  target  during  the  missile  flyout. 

These  factors  are  compared  in  Table  4. 1.2. 3-1.  This  table  shows  that  the  Parametric  Study 
Mission  run  which  best  satisfied  all  of  the  above  criteria  was  Run  #12.  Thus,  Run  #12  was  used 
for  the  quantitative  verification. 
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Table  4.1.2.3-1.  Comparison  of  Parametric  Study  Mission  V2  Runs  to  LPN-15 


Run 

Delta 
Aspect  (°) 

Delta  Lead 
(°) 

Delta  Flare 
Time  (sec) 

Delta  TOF 
(sec) 

Total 

Deltas 

Target  Alt. 
Variation  (ft) 

9 

4.45 

1.26 

0.60 

0.30 

6.61 

61.6 

10 

1.52 

2.78 

1.48 

0.21 

5.99 

101.8 

12 

3.44 

0.09 

0.38 

0.08 

3.99 

40.7 

18 

5.10 

1.22 

1.92 

0.54 

8.78 

301.6 

19 

4.87 

2.39 

1.66 

0.40 

9.32 

241.4 

22 

5.72 

3.72 

0.27 

0.15 

9.86 

81.1 

The  launch  conditions  and  target  trajectory  from  Run  #12  were  used  as  inputs  to  SIMLAB 
standalone  runs  for  the  quantitative  validation  method.  The  SIMLAB  standalone  runs  were 
performed  after  correcting  the  target  initialization  error  in  the  SIMLAB  reference  frame  (see 
Section  4.1.1 .3).  The  missile  flyouts  for  20  SIMLAB  standalone  runs  were  used  to  determine  the 
envelope  of  bounding  values  (as  described  in  Fig.  4.1.2.2-1).  The  envelopes  were  then  compared 
to  the  missile  flyout  from  Run  #12,  as  shown  in  Figures  4.1.2.3-14  through  4.1.2.3-16.  These 
figures  show  the  following: 

-  The  missile  flyouts  during  the  Parametric  Study  Mission  were  in  error  because  of  the 
target  initialization  error  in  the  SIMLAB  reference  frame.  This  is  shown  most  clearly  in 
the  “God’s-eye”  view  (Fig.  4.1.2.3-14). 

-  The  first  side  view  (Fig.  4.1.2.3-15)  shows  a  “dip”  and  recovery  in  the  missile  trajectory 
for  Run  #12  compared  to  the  SIMLAB  standalone  envelope.  This  apparently  was  due  to 
errors  in  initializing  the  missile  yaw  and  pitch  for  the  linked  runs,  which  were  not 
discovered  until  after  the  linked  runs,  but  before  the  SIMLAB  standalone  runs. 

There  was  good  agreement  between  the  TOFs  for  Run  #12  and  those  for  the  SIMLAB  standalone 
runs.  The  TOF  for  Run  #12  was  within  about  1%  of  the  mean  TOF  for  the  20  SIMLAB 
standalone  runs.  This,  along  with  the  qualitative  comparison  to  LPN-15,  indicates  that  the  missile 
flyout  for  Rim  #12  was  valid  for  the  target  representation  in  the  SIMLAB  reference  frame. 

CONCLUSION .  The  LSP  results  for  missile  flyout  were  invalid  because  of  incorrect 
representation  of  the  target  to  the  missile  simulation  and  errors  in  initializing  the  missile 
orientation.  However,  the  initialization  errors  have  since  been  fixed. 
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Figure  4.1.2.3-8b.  Missile  Flyout  for  Run  #9  (11/19/96)  Compared  to  Envelope  of  SIMLAB  Standalone  Runs 

(using  exact  LPN-15  launch  conditions)  -  Side  View  #1 


Run  #9  -  Missile 
Run  #9  -  Target 

Standalone  Runs  -  Missile  Envelope 
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Figure  4.1.2.3-8c.  Missile  Flyout  for  Run  #9  (11/19/96)  Compared  to  Envelope  of  SIMLAB  Standalone  Runs 

(using  exact  LPN-15  launch  conditions)  -  Side  View  #2 


—♦—Run  #10  -  Missile 
Run  #1 0  -  T  arget 

^—Standalone  Runs  -  Missile  Envelope 
- Standalone  Runs  -  Target 
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Figure  4.1.2.3-9c.  Missile  Flyout  for  Run  #10  (11/19/96)  Compared  to  Envelope  of  SIMLAB  Standalone  Runs 

(using  exact  LPN-15  launch  conditions)  -  Side  View  #2 
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Figure  4.1.2.3-10c.  Missile  Flyout  for  Run  #12  (11/19/96)  Compared  to  Envelope  of  SIMLAB  Standalone  Runs 

(using  exact  LPN-15  launch  conditions)  -  Side  View  #2 
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Figure  4.1.2.3-lla.  Missile  Flyout  for  Run  #18  (11/19/96)  Compared  to  Envelope  of  SIMLAB  Standalone  Runs 

(using  exact  LPN-15  launch  conditions)  -  “God’s-Eye”  View 


—♦—Run  #18  -  Missile 
-•—Run  #18  -  Target 
^—Standalone  Runs  -  Missile  Envelope 
— — •  Standalone  Runs  -  Target 
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Figure  4.1.2.3-llc.  Missile  Flyout  for  Run  #18  (11/19/96)  Compared  to  Envelope  of  SIMLAB  Standalone  Runs 

(using  exact  LPN-15  launch  conditions)  -  Side  View  #2 


Run  #1 9  -  Missile 
Run  #19 -Target 

Standalone  Runs  -  Missile  Envelope 


Figure  4.1.2.3-12b.  Missile  Flyout  for  Run  #19  (11/19/96)  Compared  to  Envelope  of  SIMLAB  Standalone  Runs 

(using  exact  LPN-15  launch  conditions)  -  Side  View  #1 
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Figure  4.1.2.3-12c.  Missile  Flyout  for  Run  #19  (11/19/96)  Compared  to  Envelope  of  SIMLAB  Standalone  Runs 

(using  exact  LPN-15  launch  conditions)  -  Side  View  #2 


Figure  4.1.2.3-13b.  Missile  Flyout  for  Run  #22  (11/19/96)  Compared  to  Envelope  of  SIMLAB  Standalone  Runs 

(using  exact  LPN-15  launch  conditions)  -  Side  View  #1 
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Figure  4.1.2.3-13c.  Missile  Flyout  for  Run  #22  (11/19/96)  Compared  to  Envelope  of  SBMLAB  Standalone  Runs 

(using  exact  LPN-15  launch  conditions)  -  Side  View  #2 


■Standalone  Runs  -  Missile  Envelope 
Standalone  Runs  -  Target 
-Run  #12 -Missile 


Figure  4.1.2.3-14.  Missile  Flyout  for  Run  #12  (11/19/96)  Compared  to  Envelope  of  SIMLAB  Standalone  Runs  (using  exact 

Run  #12  launch  conditions)  -  “God’s-Eye”  View 
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Figure  4.1.2.3-15.  Missile  Flyout  for  Run  #12  (11/19/96)  Compared  to  Envelope  of  SIMLAB  Standalone  Runs  (using  exact 

Run  #12  launch  conditions)  -  Side  View  #1 
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Figure  4.1.2.3-16.  Missile  Flyout  for  Run  #12  (11/19/96)  Compared  to  Envelope  of  SIMLAB  Standalone  Runs  (using  exact 

Run  #12  launch  conditions)  -  Side  View  #2 


4.1.2.4  Validation  Summary 

The  results  for  the  validation  objective  are  summarized  as  follows: 

-  The  SIMLAB  standalone  simulation  of  the  missile  flyout  for  the  LPN-15  conditions  was 
valid. 

-  A  problem  with  the  missile  lofting  above  the  shooter  altitude  was  observed  during  the 
Mission  Rehearsal  and  the  V&V  Mission.  The  lofting  problem  was  fixed  for  the 
Parametric  Study  Mission. 

-  The  planned  quantitative  validation  technique  based  on  comparing  the  missile  flyouts  from 
the  linked  runs  to  the  envelopes  resulting  from  Monte  Carlo  sampling  of  launch  conditions 
had  to  be  abandoned.  It  was  incapable  of  identifying  many  of  the  invalid  lofting  missile 
trajectories. 

-  The  validation  approach  was  modified  to  include  both  qualitative  and  quantitative 
validation  methods. 

—  The  qualitative  method  compared  the  shape  of  the  missile  trajectory  to  that  for  LPN- 
15  by  looking  for  the  following  qualitative  features: 

—  An  initial  straight  “safe-separation”  segment. 

—  A  distinct  guidance  correction  at  the  end  of  the  “safe-separation  segment. 

—  Continual  and  smooth  closing  on  the  target  with  no  gain  in  missile  altitude. 

—  This  method  identified  the  missile  lofting  as  an  invalid  flyout. 

-  The  quantitative  method  involved  SIMLAB  standalone  runs  using  the  linked  result  in 
which  the  launch  conditions  most  closely  matched  those  of  LPN-15. 

—  On  each  trial,  the  exact  launch  conditions  of  the  best  linkedrun  were  used. 

—  The  results  of  20  missile  flyouts  were  used  to  define  an  envelope  of  valid  flyouts. 
—  The  flyout  of  the  linked  run  was  validated  by  comparing  it  to  the  envelope  to 
determine  if  the  missile  trajectory  was  entirely  within  the  envelope. 

-  Applying  the  qualitative  method  to  the  Parametric  Study  Mission  V2  runs  showed  that  the 
missile  flyouts  were  valid  for  the  target  representation  in  the  SIMLAB  reference  frame. 

-  Applying  the  quantitative  method  to  the  best  of  the  Parametric  Study  Mission  V2  runs 
showed  that  the  missile  flyouts  from  the  linked  runs  were  invalid  because  the  target 
representation  in  the  SIMLAB  reference  frame  was  in  error. 

~  Target  representation  errors  resulted  in  the  missile  flying  against  a  significantly 
different  target  trajectory  than  was  generated  by  the  WSIC. 

-  Errors  in  initializing  the  target  and  missile  in  the  SIMLAB  reference  frame  were  not 
discovered  until  after  the  linked  runs  were  completed  and  have  since  been  fixed. 

—  During  the  missions,  quick-look  qualit  ative  validation  of  the  SIMLAB  results  was 
performed  by  using  plots  from  the  SIMLAB  simulation.  These  plots  were 
successful  in  identifying  the  missile  lofting  and  latitude  divergence  problems,  but 
not  the  target  initialization  error.  The  latter  was  not  discovered  until  the  missile 
and  target  trajectories  were  plotted  from  PDU  data  after  the  testing  was  over. 

~  Validity  of  the  missile  flyout  can  be  further  improved  by  more  accurate  SIMLAB 
integration  of  the  target  velocity  to  determine  target  position. 
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4.2  Test  Objective  2:  Assess  utility  of  LSP  ADS  configuration  for  parametric  studies 

This  objective  evaluates  a  potential  benefit  of  the  LSP  ADS  configuration  to  AIM-9  testing:  the 
ability  to  conduct  parametric  studies.  The  assessment  addressed  two  questions:  (1)  can  valid 
parametric  studies  involving  countermeasures  (CM)  effectiveness  be  conducted  with  this  ADS 
configuration?  (2)  if  so,  is  there  utility  in  using  this  ADS  configuration  for  such  studies?  The  first 
question  was  addressed  by  evaluating  the  ability  to  repeat  a  given  scenario  with  either  no  changes 
or  with  a  single  parameter  varying.  The  second  question  was  addressed  by  evaluating  the  cost 
and  efficiency  of  executing  the  parametric  studies  using  the  LSP  ADS  configuration. 

4.2.1  Parametric  Study  Test  Method 

The  Parametric  Study  Mission  was  designed  to  accomplish  this  test  objective  and  was  to  utilize 
variations  in  the  live  fire  test  baseline,  LPN-15.  In  this  mission,  the  engagement  was  to  begin  with 
the  same  initial  conditions  as  in  LPN-15  (Fig.  4. 1.2. 1-1).  The  difference  was  to  be  that  the  target 
would  now  receive  a  missile  warning  detection  cue  (either  at  or  after  launch)  and  employ 
countermeasures  based  on  that  cue  (rather  than  at  a  preset  time,  as  in  LPN-15).  The 
countermeasures  were  to  always  involve  ejection  of  flares,  but  could  also  include  evasive 
maneuvers. 

Features  of  the  Parametric  Study  Mission  were  to  be  as  follows: 

-  This  mission  was  to  utilize  the  same  initial  engagement  geometry  as  the  V&V  Mission. 

The  V&V  profile  was  to  be  modified  by  changing  the  flare  release  time  and  by  executing 
an  evasive  maneuver  (in  some  profiles)  after  the  missile  was  launched.  The  values  for  the 
flare  release  and  evasive  maneuver  times  were  to  be  0,  2,  or  4  seconds  after  missile  launch. 

The  mission  was  to  begin  with  repetitions  of  the  verification  and  the  validation  passes 
from  the  V&V  Mission  to  ensure  the  test  configuration  was  working  properly.  Note  that 
the  V&V  Mission  did  not  utilize  warning  cues  to  dispense  the  flares  (they  were  dispensed 
at  a  preset  time  during  the  engagement). 

-  Half  of  the  runs  for  each  profile  were  to  be  manually  flown  by  the  simulator  pilots  on  the 
first  day  of  the  mission.  The  manual  trial  for  each  profile  which  best  matched  the  planned 
profile  was  to  be  automatically  replayed  on  the  second  day  of  the  mission  for  the 
remaining  runs.  The  automatic  replay  runs  were  executed  as  follows: 

-  The  shooter  and  target  simulators  replayed  their  respective  recorded  output  from  one 
of  the  manual  linked  runs.  Each  aircraft  simulator  was  started  manually  by  local 
operators  when  directed  by  the  test  controller.  In  this  mode  there  were  no  pilots  flying 
or  controlling  the  shooter  and  target  simulators  (except  for  manual  trigger  squeeze  by 
the  shooter  and  manual  flare  release).  The  SIMLAB  (missile)  was  driven  in  each  run 
by  inputs  from  the  shooter  and  target.  Control  of  shooter  and  target  trajectories  and 
timed  events  (except  for  trigger  squeeze  and  flare  release)  were  all  embedded  in  the 
playback  data. 
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-  The  countermeasures  (flare  release  and  target  maneuver)  were  to  be  executed  when  the 
warning  cue  was  received  by  the  target. 

-  Automatic  flare  release  was  to  initiated  by  automatic  transmission  of  a  Fire  (Flare)  PDU 
from  the  WSIC  to  the  SIMLAB.  Upon  receipt  of  this  PDU,  the  SIMLAB  flare  simulation 
was  to  be  started. 

-  In  all  passes,  the  target  aircraft  was  already  in  a  3.6  g  turn  at  missile  launch  (see  Fig. 
4. 1.2. 1-1).  In  the  runs  involving  the  evasive  maneuver,  the  target  pilot  was  to  increase  the 
turn  rate  by  at  least  3.4  g  to  7  g,  or  more,  upon  receipt  of  the  warning  cue. 

4.2.2  Parametric  Study  Analysis  Method 

Data  from  the  manual  runs  performed  on  the  first  day  of  the  Parametric  Study  Mission  were 
analyzed  to  determine  which  run  best  achieved  each  of  the  desired  profiles.  Criteria  for  this 
determination  were:  (1)  smallest  differences  between  initial  launch  conditions  achieved  in  the  run 
and  those  for  LPN-15,  (2)  smallest  differences  between  target  profile  (position,  velocity,  and 
acceleration  versus  time)  achieved  in  the  run  and  the  desired  profile,  and  (3)  smallest  difference 
between  flare  release  time  achieved  in  the  run  and  the  desired  value.  The  differences  needed  for 
the  selection  were  computed  as  follows: 

-  Initial  launch  condition  differences 

—  Each  launch  condition  parameter,  as  determined  by  the  SIMLAB  missile  simulation, 
was  compared  to  the  corresponding  parameter  from  LPN-15. 

—  According  to  the  LSP  TAP,  the  percent  difference  between  the  SIMLAB  value  and 
the  LPN-15  value  was  to  be  computed  as  (the  desired  value  is  the  LPN-15  value  in 
this  case): 


Percent  Difference  = 


(Actual  Value)  -  (Desired  Value) 
Desired  Value 


(3) 


—  However,  Equation  3  was  modified  to  simply  compute  the  difference.  This  was  done 
because  the  raw  differences  could  be  compared  directly  to  the  shot  box  tolerances  and 
because  some  of  the  parameters  (e.g.  altitudes)  had  large  values  and  the  percent 
differences  would  have  been  arbitrarily  small.  Hence,  the  following  equation  was  used 
instead  of  Equation  3 : 

Difference  =  (Actual  Value)  -  (Desired  Value)  (4) 


-  Target  profile  differences  during  missile  flyout 

—  Five  profiles  were  to  be  separately  compared:  x-position  vs.  time,  y-position  vs.  time, 
z-position  vs.  time,  velocity  magnitude  vs.  time,  and  acceleration  magnitude  vs.  time, 
—  This  was  modified  to  only  compare  altitude  vs.  tune,  velocity  magnitude  vs.  time, 
and  acceleration  vs.  time.  This  was  changed  because  differences  in  the  target 
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profile  during  missile  flyout  primarily  depended  on  differences  in  launch  conditions 
(only  the  target  location  and  orientation  relative  to  the  shooter  and  missile 
mattered  during  the  missile  flyout,  not  the  target’s  absolute  location)  and  because 
the  key  features  the  desired  target  trajectory  were  constant  altitude,  velocity,  and 
acceleration. 

—  The  percent  difference  between  the  actual  value  achieved  on  the  run  and  the  desired 
value  was  to  be  computed  at  each  0.1  second  increment  of  time  using  Equation  3  (data 
were  to  be  interpolated,  as  needed,  to  determine  the  values  at  each  time  increment). 
—  This  was  modified  to  only  compute  the  difference,  not  the  percent  difference,  using 

Equation  4. 

—  Rather  than  interpolate  data  to  determine  values  at  0.1  second  time  increments, 
data  from  the  target  entity  state  PDUs  logged  at  the  SIMLAB  DIS  logger  were 
averaged  during  the  missile  flyout.  Although  the  PDUs  were  not  logged  at 
constant  rates,  this  approach  provided  sufficient  accuracy  for  determining  average 
differences. 

-  Flare  release  time  difference 

-  The  percent  difference  between  the  actual  value  achieved  o  n  the  run  and  the  desired 
value  was  to  be  computed  using  Equation  3. 

—  This  was  modified  to  only  compute  the  difference,  not  the  percent  difference,  using 
Equation  4. 


Data  from  the  automatic  replay  runs  were  analyzed  to  determine  the  run-to-run  variations  for  each 
profile.  Differences  for  initial  launch  conditions,  target  profiles,  and  flare  release  time  were 
computed  as  above  using  the  data  from  the  replay  runs  of  each  profile.  The  mean  and  standard 
deviation  of  the  differences  for  all  runs  of  each  profile  were  computed. 

The  average  number  of  runs  per  hour  for  the  entire  Parametric  Study  Mission  was  computed 
using  the  total  number  of  runs  executed  divided  by  the  total  time  that  the  LSP  configuration  was 
operational.  Statistics  for  each  parametric  variation  (i.e.,  profile)  were  also  computed  as  follows: 

-  The  set  up  time  for  each  profile  was  determined  by  subtracting  the  time  when  set  up  was 
started  from  the  time  when  set  up  was  completed.  The  set  up  time  was  separately 
computed  for  manual  runs  (Day  1  of  the  mission)  and  automatic  runs  (Day  2  of  the 
mission). 

-  The  runs  per  hour  for  each  profile  was  determined  by  dividing  the  total  number  of  runs  by 
the  total  time  to  execute  the  runs.  The  latter  was  determined  by  subtracting  the  time  when 
die  first  run  of  the  profile  was  begun  from  the  time  when  the  last  run  of  the  profile  was 
completed.  The  runs  per  hour  was  separately  computed  for  manual  runs  and  automatic 
runs. 

The  cost  per  run  was  computed  by  dividing  the  total  cost  of  the  Parametric  Study  Mission  by  the 
total  number  of  valid  runs  executed  (a  valid  run  was  one  which  produced  valid  SIMLAB  results) 
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4.2.3  Parametric  Study  Results 

The  Parametric  Study  Mission  test  matrix,  as  given  in  the  LSP  TAP ,  could  not  be  executed.  Due 
to  invalid  missile  flyouts  in  the  V&V  Mission  (10/29/96),  additional  runs  during  the  Parametric 
Study  Mission  (11/19/96)  had  to  be  devoted  to  V&V.  Also,  AIM-9  hardware  problems  at  the 
SIMLAB  prevented  accomplishing  all  the  planned  runs  for  the  Parametric  Study  Mission.  In 
particular,  none  of  the  parametric  runs  in  which  the  flares  simulation  was  triggered  by  the  WSIC 
were  successfully  completed.  Rather,  only  the  V2  profile  (which  replicated  LPN-15)  was 
successfully  performed. 

Because  of  these  problems  during  the  Parametric  Study  Mission,  data  for  evaluating  Test 
Objective  2  was  taken  from  the  Mission  Rehearsal  (8/27/96  and  8/28/96).  There  are  several 
reasons  for  using  these  data: 

-  The  rehearsal  practiced  execution  of  the  V2  profile  using  both  manual  and  automatic 
replay  methods.  As  discussed  below,  this  was  the  only  testing  period  during  which  the 
automatic  replay  method  was  used. 

A  relatively  large  number  of  runs  were  successfully  completed,  and  the  aircraft  simulator 
pilots  had  the  opportunity  to  leam  their  cues  for  reliably  reproducing  the  LPN-15  launch 
conditions. 

Although  the  Mission  Rehearsal  only  used  the  VI  and  V2  profiles,  the  results  were 
adequate  for  evaluating  Test  Objective  2.  The  objective  was  to  evaluate  differences 
between  the  actual  and  planned  profile  parameters  using  Equation  4,  and  this  can  be 
satisfied  by  using  only  a  single  planned  profile  (LPN-15),  rather  than  a  number  of  profiles. 

The  analysis  results  are  given  separately  for  the  manual  and  the  automatic  replay  methods. 

4.2.3.1  Manual  Runs  Results 

Launch  Conditions  .  The  launch  conditions  obtained  in  each  successful  V2  run  (successful  runs 
were  those  in  which  the  missile  was  launched,  even  if  the  launch  conditions  were  outside  the  shot 
box  or  the  missile  flyout  was  incomplete)  were  compared  to  those  for  LPN-15  and  the  differences 
were  computed  using  Equation  4.  The  manual  runs  were  executed  over  two  days  during  the 
Mission  Rehearsal,  and  results  are  given  separately  for  each  day.  The  resulting  differences 
between  the  actual  launch  conditions  and  the  LPN-15  values  are  given  for  each  manual  run  in 
Figures  4.2.3. 1-1  through  4.2.3. 1-9  (NOTE:  the  trial  numbers  in  these  figures  are  consecutive 
for  the  runs  selected  and  are  not  the  same  as  the  run  numbers).  These  figures  also  indicate  the 
shot  box  tolerances  for  each  parameter  and  show  a  second  degree  polynomial  fit  to  the  data.  The 
mean  and  standard  deviations  for  the  differences  in  each  launch  condition  parameter  are 
summarized  in  Table  4.2.3. 1-1. 


Table  4.2.3.1-1.  Differences  Between  Actual  and  Desired  Launch  Conditions  for  Manual 

Runs. 


Launch  Parameter 

i 
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Shooter  Velocity 

11.05  ±8.05  ft/s 

23.18  ±17.01  ft/s 

15.09  ±  12.98  ft/s 

Shooter  Altitude 

40.5  ±  134.1  ft 

-3.03  ±  127.02  ft 

26.0  ±132.1  ft 

Target  Velocity 

12.73  ±  4.45  ft/s 

15.01  ±5.30  ft/s 

13.50  ±4.82  ft/s 

Target  Altitude 

76.6  ±53.0  ft 

56.1  ±59.5  ft 

69.8  ±55.5  ft 

Difference  in 
Altitudes 

-36.1  ±151.7  ft 

-59.1  ±  127.3  ft 

-43.8  ±  143.1  ft 

Launch  Range 

776.4  ±  292.4  ft 

201.7  ±426.4  ft 

584.8  ±435.2  ft 

Target  Aspect  Angle 

4.83  ±2.56  deg 

4.37  ±  2.44  deg 

Lead  Angle 

-2.61  ±1.58  deg 

Kfclil'ldriHi 

-2.72  ±1.41  deg 

Flare  Initiation  Time 

0.11  ±  1.58  sec 

-0.13  ±  0.38  sec 

0.03  ±1.30  sec 

The  following  conclusions  are  obtained  from  the  figures  and  table: 

-  The  distribution  of  results  from  the  manual  runs  was  significantly  tighter  than  the  shot  box 
tolerances  for  all  launch  parameters.  This  may  indicate  that  the  shot  box  was  wider  than 
necessary,  but  does  show  good  reproducibility  of  launch  conditions  by  the  pilots. 

-  In  general,  launch  conditions  from  the  manual  runs  showed  bias.  This  was  indicated  by 
the  mean  values  of  the  differences  differing  significantly  from  zero  and  by  the  means  being 
significantly  larger  than  the  standard  deviations  (except  for  altitudes  and  flare  initiation 
time).  This  may  have  been  peculiar  to  the  cues  used  by  the  pilots  to  achieve  the  desired 
launch  conditions.  Nevertheless,  the  mean  and  standard  deviation  values  in  Table  4.2.3. 1- 
1  were  all  well  within  the  shot  box. 

-  The  polynomial  fits  to  the  data  points  in  the  figures  show  that  most  parameters  did  not 
display  a  definite  trend  toward  near-zero  differences.  The  exception  was  the  flare 
initiation  time.  Part  of  the  reason  for  this  was  that  the  pilot  developed  cues  for  the  launch 
conditions  during  dry  runs  prior  to  the  Mission  Rehearsal,  but  the  proper  manual  flare 
initiation  time  was  determined  by  trial  and  error  during  the  mission. 

-  Results  for  the  individual  parameters  are  as  follows: 

—  Velocities  of  the  shooter  and  target  showed  comparable,  small  positive  biases  (-30% 
of  upper  shot  box  limit).  Comparing  Figures  4.2.3. 1-1  and  4.2.3. 1-3  shows  that  the 
target  was  better  able  to  maintain  about  the  same  velocity  on  each  run.  This  was 
because  the  target  was  executing  a  level  constant  velocity  turn,  but  the  shooter  had  to 
lower  his  nose  to  boresight  on  the  target  at  launch.  This  caused  the  shooter  to  lose 
altitude  and  gain  velocity  prior  to  launch  with  more  run-to-run  velocity  variations. 
The  inability  of  the  target  pilot  to  execute  a  perfectly  level  turn  contributed  to  his  run- 
to-run  velocity  variations. 

—  The  shooter  altitude  was  the  only  aircraft-related  launch  condition  that  had  a  mean 
value  smaller  than  the  standard  deviation.  This  indicates  essentially  no  bias  in  this 
parameter.  Note  the  relatively  large  run-to-run  variations  caused  by  the  shooter  losing 
altitude  at  launch,  as  discussed  above. 

-  The  target  altitude,  like  the  target  velocity,  showed  a  small  positive  bias  (-14%  of 
upper  shot  box  limit).  The  run-to-run  variations  in  the  target  altitude  are  significantly 
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smaller  that  for  the  shooter  altitude.  This  is  to  be  expected,  since  the  target  was 
attempting  to  execute  a  flat  turn  while  the  shooter  was  losing  altitude. 

-  The  difference  between  the  shooter  and  target  altitudes  is  given  because  only  the 
altitude  difference  really  mattered  for  the  engagement.  Variations  in  the  altitude 
difference  were  dominated  by  the  shooter  altitude  variations. 

-  The  launch  range  displayed  a  bias  toward  longer  ranges  than  the  LPN-15  value, 
although  the  trend  in  the  second  day  data  was  toward  the  LPN-15  value.  Note  that  2 
of  the  48  trials  (~4%)  exceeded  the  upper  shot  box  limit.  The  shooter  was  closing  on 
the  target  prior  to  launch,  and  the  start  of  the  engagement  had  to  be  set  up  at  a  range 
longer  than  the  LPN-15  launch  range  in  order  to  be  able  to  achieve  the  LPN-15  value 
at  launch.  This  bias  shows  that  the  shooter  was  launching  the  missile  too  early  for  this 
parameter,  in  general. 

-  The  target  aspect  angle  displayed  a  bias  toward  larger  angles  than  the  LPN-15  value. 
The  target  was  executing  a  right  turn  in  front  of  the  shooter  prior  to  launch,  so  that  the 
aspect  angle  was  increasing  with  time.  This  bias  shows  that  the  shooter  was  launching 
the  missile  too  late  for  this  parameter,  in  general. 

—  The  lead  angle  displayed  a  bias  toward  lag  angles  (the  desired  LPN-15  value  was  0  °). 
Note  that  3  of  the  48  trial  (~6%)  exceeded  the  lower  shot  box  limit.  The  shooter 
turned  at  launch  to  boresight  on  the  target,  starting  from  a  lag  angle.  This  bias  shows 
that  the  shooter  was  launching  the  missile  too  early  for  this  parameter,  in  general. 

—  The  flare  initiation  time  showed  a  mean  value  ve  ry  close  to  the  LPN-15  value.  After 
the  sixth  trial,  the  difference  on  each  trial  was  ~1  second,  or  less. 

These  results  support  the  conclusion  that  the  manual  method  for  replicating  a  given  profile  (LPN- 
15  in  this  case)  resulted  in  very  good  run-to-run  reproducibility  of  the  launch  conditions. 
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Figure  4.2.3.1-1.  Shooter  Velocity  Relative  to  LPN-15  (Manual  Trials) 
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Figure  4.2.3.1-2.  Shooter  Altitude  Relative  to  LPN-15  (Manual  Trials) 
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Figure  4.2.3.1-3.  Target  Velocity  Relative  to  LPN-15  (Manual  Trials) 
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Figure  4.2.3.1-4.  Target  Altitude  Relative  to  LPN-15  (Manual  Trials) 
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Figure  4.2.3.1-5.  Shooter  and  Target  Altitude  Difference  Relative  to  LPN-15  (Manual  Trials) 
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Figure  4.2.3.1-6.  Launch  Range  Relative  to  LPN-15  (Manual  Trials) 
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Figure  4.2.3.1-8.  Lead  Angle  Relative  to  LPN-15  (Manual  Trials) 


Figure  4.2.3.1-9.  Flare  Initiation  Time  Relative  to  LPN-15  (Manual  Trials) 


Target  Profile  During  Missile  Flvout  .  The  entity  state  PDU  data  for  the  target  profile  obtained 
in  each  successful  and  complete  V2  run  (these  runs  were  those  in  which  the  missile  was  launched 
and  the  missile  flyout  was  complete,  even  if  the  launch  conditions  were  outside  the  shot  box)  were 
compared  to  those  for  LPN-15  and  the  differences  were  computed  for  each  target  entity  state 
PDU  using  Equation  4.  The  target  parameters  of  interest  were  the  target  altitude  (the  target  was 
to  maintain  a  constant  altitude  of  10,387  ft  during  the  missile  flyout),  the  target  velocity  (the 
target  was  to  maintain  a  constant  velocity  of  759  ft/s  during  the  missile  flyout),  and  the  target 
acceleration  (the  target  was  to  execute  a  constant  3.6  g  right  turn  during  the  missile  flyout).  The 
mean  and  standard  deviation  values  of  the  differences  in  these  parameters  are  given  in  Table 
4.2.3. 1-2  (Day  1)  and  Table  4.2.3. 1-3  (Day  2)  for  each  manual  trial  analyzed  (as  before  the  trials 
are  numbered  consecutively,  the  trial  number  is  not  the  same  as  the  run  number,  and  only  the 
manual  trials  resulting  in  complete  missile  flyouts  and  data  logging  are  shown). 


Table  4.2.3.1-2.  Target  Parameters  During  Missile  Flyout  Relative  to  LPN-15  Values 

(Manual  Trials,  Day  1) 


Trial  # 

Delta  Altitude  (ft) 

Delta  Velocity  (ft/s) 

Delta  Acceleration  (g) 

1 

246.2  ±  64.8 

0.53  ±1.11 

-0.07  ±0.15 

2 

32.7  ±  11.3 

17.41  ±1.75 

0.25  ±0.17 

3 

79.6  ±6.4 

23.09  ±4.31 

-0.10  ±0.09 

6 

118.1  ±17.2 

16.4  ±2.79 

-0.09  ±  0.06 

8 

161.2  ±20.2 

13.45  ±0.81 

0.06  ±  0.08 

9 

164.0  ±22.3 

10.51  ±1.86 

0.05  ±0.15 

14 

62.4  ±  34.7 

17.99  ±0.91 

-0.04  ±  0.07 

15 

45.8  ±15.4 

15.37  ±0.51 

0.05  ±  0.05 

16 

12.1  ±22.8 

9.34  ±2.37 

0  ±  0.19 

17 

56.2  ±38.5 

11.04  ±1.91 

-0.05  ±0.17 

18 

102.7  ±20.5 

8.93  ±  1.24 

-0.1  ±0.15 

19 

55.9  ±45.6 

10.90±  1.11 

-0.26  ±0.14 

20 

-16.6  ±26.3 

2.15  ±2.04 

0.08  ±0.03 

21 

66.9  ±  46.7 

13.29  ±0.81 

0.04  ±0.04 

22 

25.3  ±26.3 

15.82  ±0.96 

0.09  ±0.10 

23 

44.8  ±35.1 

15.88  ±1.17 

-0.03  ±  0.08 

24 

62.6  ±21.3 

8.72  ±  1.78 

-0.03  ±  0.06 

25 

149.7  ±27.4 

7.09  ±1.67 

-0.05  ±  0.07 

26 

99.0  ±38.0 

8.72  ±0.71 

-0.06  ±0.01 

27 

5.0  ±48.8 

14.95  ±1.12 

-0.09  ±  0.07 

28 

-28.8  ±  20.6 

17.46  ±0.77 

-0.24  ±  0.06 

29 

190.1  ±16.2 

13.56  ±0.79 

-0.01  ±0.09 

30 

-25.4  ±31.3 

4.59  ±  1.06 

-0.02  ±  0.05 

31 

80.2  ±29.8 

7.96  ±1.88 

0  ±  0.12 

32 

47.7  ±  6.8 

14.47  ±2.33 

0  ±  0.18 

Averages 

73.5  ±28.2 

11.98  ±  1.51 

-0.02  ±0.10 
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Table  4.2.3.1-3.  Target  Parameters  During  Missile  Flyout  Relative  to  LPN-15  Values 

(Manual  Trials,  Day  2) 


Trial  # 

Delta  Altitude  (ft) 

Delta  Velocity  (ft/s) 

Delta  Acceleration  (g) 

33 

36.4  ±  27.7 

14.91  ±0.47 

-0.12  ±0.07 

34 

48.5  ±29.8 

6.87  ±1.85 

-0.16  ±  0.16 

35 

-69.0  ±  39.2 

18.95  ±1.01 

-0.05  ±0.17  1 

36 

76.0  ±  10.4 

14.51  ±0.39 

0.05  ±  0.09 

38 

0.6  ±34.6 

21.38  ±1.00 

-0.08  ±  0.04 

39 

183.2  ±23.6 

10.09  ±0.35 

0.04  ±0.10 

40 

77.9  ±23.0 

14.27  ±0.63 

-0.03  ±  0.08 

42 

104.6  ±31.9 

23.10  ±1.21 

-0.27  ±0.11 

43 

-137.2  ±23.0 

17.88  ±2.39 

-0.07  ±0.17 

44 

58.5  ±9.4 

13.00  ±0.68 

0.02  ±  0.04 

46 

-20.1  ±  13.2 

3.21  ±3.06 

0.09  ±0.12 

Averages 

32.7  ±  24.2 

14.38  ±1.19 

-0.05  ±0.10 

Averages 
Both  Days 

61.0  ±27.0 

12.71  ±1.41 

-0.03  ±0.10 

The  mean  values  of  the  differences  in  the  target  parameters  for  each  trial  are  plotted  in  Figures 
4.2.3.1-10  through  4.2.3.1-12.  These  figures  also  show  a  second  degree  polynomial  fit  to  the 
data.  The  following  conclusions  are  obtained  from  the  figures  and  tables: 

-  Comparing  Tables  4.2.3.1-2  and  4.2.3.1-3  with  Table  4.2.3.1-1  shows  that  the  mean 
values  of  the  differences  in  die  target  altitudes  and  velocities  during  the  missile  flyouts 
(61.0  ft  and  12.71  ft/s)  were  comparable  to  the  mean  values  for  the  corresponding 
parameters  at  launch  (69.8  ft  and  13.50  ft/s).  Also,  the  spreads  of  the  mean  values  during 
the  missile  flyouts  (±75.8  ft  and  ±5.53  fl/s)  were  comparable  to  the  spreads  for  the 
corresponding  parameters  at  launch  (  ±55.5  ft  and  ±4.82  fl/s).  However,  the  variations  in 
the  target  altitude  and  velocity  during  a  single  trial  (  ±27.0  ft  and  ±1.41  ft/s)  were 
significantly  less  than  the  spreads  for  the  corresponding  launch  condition  parameters.  In 
other  words,  the  target  was  able  to  maintain  a  relatively  constant  altitude  and  velocity 
during  a  trial,  even  though  the  mean  values  varied  from  run-to-run. 

-  The  target  was  able  to  maintain  a  constant  3.6  g  turn  to  within  ±0.1  g.  This  was  much 
better  than  the  criterion  given  in  the  LSP  TAP  for  acceptable  trials  (b0.5  g). 

These  results,  along  with  those  for  the  launch  conditions,  support  the  conclusion  that  the  manual 
method  for  repheating  a  given  profile  (LPN-15  in  this  case)  resulted  in  very  good  run-to-run 
reproducibility  of  the  entire  aircraft  portion  of  the  scenario.  (Note  that  LPN-15  is  a  fairly  dynamic 
scenario  (target  turning,  shooter  closing  on  target  and  descending),  so  that  it  was  not  possible  for 
the  pilots  to  precisely  match  all  the  LPN-15  conditions  simultaneously.)  Hence,  the  manual 
method  is  able  to  effectively  support  parametric  studies. 
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Figure  4.2.3.1-10.  Mean  Target  Altitude  During  Missile  Flyout  Relative  to  LPN-15  Value  (Manual  Trials) 


Figure  4.2.3.1-11.  Mean  Target  Velocity  During  Missile  Flyout  Relative  to  LPN-15  Value  (Manual  Trials) 


Time  Refluired  for  Manna!  Runs  .  The  total  time  required  to  execute  54  manual  V2  replication 
runs  was  4  hours  which  averaged  13.5  runs/hr  (~5  min.  between  runs).  This  included  the  set  up 
time  (~1  hour)  and  the  time  for  five  VI  runs  in  preparation  for  the  V2  runs  (10  min.). 

Cost  of  the  Manual  Runs  .  The  cost  of  the  NAWCWPNS  simulation  facilities  was  $6K/hr. 
Thus,  the  total  cost  of  the  manual  runs  was  $24K. 

4.2.3.2  Automatic  Replay  Runs  Results 

The  manual  runs  from  the  first  day  of  the  Mission  Rehearsal  were  examined  to  determine  which 
run  best  matched  the  LPN-15  conditions.  No  single  manual  run  closely  matched  all  conditions 
simultaneously.  Because  of  this,  it  was  necessaiy  to  determine  which  parameter  matches  were 
most  critical  for  the  missile  flyout.  Upon  the  advice  of  the  AIM-9  expert,  the  most  critical 
parameters  were  determined  to  be  lead  angle  and  target  aspect  angle  at  launch  and  the  degree  to 
which  the  target  maintains  a  constant  altitude  and  acceleration  during  the  missile  flyout.  The  run 
which  best  matched  this  combination  of  criteria  was  determined  to  be  Manual  Trial  #15. 

Launch  Conditions  .  The  launch  conditions  obtained  in  each  successful  V2  run  (successful  runs 
were  those  in  which  the  missile  was  launched,  even  if  the  launch  conditions  were  outside  the  shot 
box  or  the  missile  flyout  was  incomplete)  were  compared  to  those  for  Manual  Trial  #15  and  the 
differences  were  computed  using  Equation  4.  The  resulting  differences  between  the  actual  launch 
conditions  and  the  Manual  Trial  #15  values  are  given  for  each  automatic  replay  run  in  Figures 
4.2.3.2-1  through  4.2. 3. 2-9  (NOTE:  the  trial  numbers  in  these  figures  are  consecutive  for  the 
runs  selected  and  are  not  the  same  as  the  run  numbers).  These  figures  also  indicate  the  shot  box 
tolerances  for  each  parameter  and  show  a  second  degree  polynomial  fit  to  the  data.  The  mean 
and  standard  deviations  for  the  differences  in  each  launch  condition  parameter  are  summarized  in 
Table  4.2.3 .2-1.  Table  4.2.3.2-1  also  compares  the  automatic  replay  results  to  the  manual  results 
from  Table  4.2.3. 1-1. 

Table  4.2.3.2-1.  Differences  Between  Actual  and  Desired  Launch  Conditions  for  Automatic 

Replay  and  Manual  Runs. 


Launch  Parameter 

Auto  V2  Runs 
vs.  Manual  Trial  #1 5 

Shooter  Velocity 

2.40  ±  0.48  ft/s 

15.09  ±  12.98  ft/s 

Shooter  Altitude 

-322.9  ±  34.9  ft 

26.0  ±132.1  ft 

Target  Velocity 

0.15  ±0.18  ft/s 

13.50  ±  4.82  ft/s 

Target  Altitude 

-17.1  ±7.35  ft 

69.8  ±55.5  ft 

Difference  in 
Altitudes 

-305.9  ±  33.4  ft 

-43.8  ±  143.1  ft 

Launch  Range 

58.0  ±539.6  ft 

584.8  ±435.2  ft 

Target  Aspect  Angle 

-2.71  ±1.98  deg 

4.37  ±  2.44  deg 

Lead  Angle 

-1.98  ±1.22  deg 

-2.72  ±1.41  deg 

Flare  Initiation  Time 

0.72  ±  0.54  sec 

0.03  ±  1.30  sec 
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The  following  conclusions  are  obtained  from  the  figures  and  table: 

The  automatic  replay  runs  did  not  perform  as  hoped.  The  objective  of  these  w  as  to 
achieve  exactly  the  same  launch  conditions  on  each  run.  However,  as  Table  4.2.3.2-1 
shows,  this  was  not  the  case;  there  should  have  been  zero  biases  and  spreads. 

-  The  non-zero  values  for  means  (biases)  and  standard  deviations  (spreads)  were  due  to 
the  manual  actions  required  during  the  automatic  replay  runs:  manual  start  of  the 
WSIC  and  WSSF  simulations,  manual  trigger  squeeze  by  the  F/A-18  WSSF  pilot,  and 
manual  flare  initiation.  As  a  result,  the  start  of  the  simulations  was  not  synchronized 
to  exactly  the  same  conditions  each  time,  and  the  trigger  squeeze  did  not  always  occur 
at  the  same  relative  time. 

-  In  comparing  the  launch  conditions  from  the  automatic  replay  runs  to  those  from  the 
manual  runs,  the  following  is  noted: 

-  Shooter  velocity,  target  velocity,  and  target  altitude  had  small  biases  and  spreads 
compared  to  the  manual  runs.  This  appears  to  be  due  to  these  parameters  remaining 
relatively  constant  throughout  the  Manual  Trial  #15,  so  that  variations  in  the  time  of 
trigger  squeeze  did  not  significantly  affect  these  parameters. 

—  Target  aspect  angle  and  lead  angle  had  about  the  same  biases  and  spreads  compared  to 
the  manual  runs. 

-  Shooter  altitude  and  flare  initiation  time  had  small  spreads  compared  to  the  manual 
runs,  but  large  biases.  This  appears  to  be  due  to  using  different  cues  for  the  manual 
actions  of  trigger  squeeze  and  flare  initiation,  compared  to  Manual  Trial  #15. 
However,  use  of  the  new  cues  in  the  automatic  replay  runs  was  consistent. 

—  Launch  range  had  a  small  bias  c  ompared  to  the  manual  runs,  but  about  the  same 
spread.  This  appears  to  be  due  to  variations  in  the  time  of  trigger  squeeze. 

-  Hence,  the  automatic  replay  runs  did  not  result  in  smaller  biases  and  spreads  in  all 
launch  conditions,  compared  to  the  manual  runs.  In  particular,  Figures  4.2.3 .2-6 
through  4.2.3. 2-8  show  significant  run-to-run  variations  in  the  following  critical  launch 
parameters:  launch  range,  target  aspect  angle,  and  lead  angle. 

The  effects  of  variations  in  the  time  of  missile  launch  are  illustrated  in  Figures  4.2.3.2-10  and 
4.2.3.2-11.  These  figures  compare  the  target  trajectory  during  the  missile  flyout  from  one  of  the 
automatic  replay  runs  with  Manual  Trial  #15.  The  following  is  noted: 

-  These  figures  show  that  the  replay  of  the  target  simulator  output  from  the  manual  trial 
overlays  the  data  from  the  manual  trial  itself.  However,  the  start  and  stop  of  the  missile 
flyout  occurred  at  different  times  during  the  automatic  replay. 

-  Figure  4.2.3.2-11  shows  a  noteworthy  feature  of  the  WSIC  target  simulator:  the  WSIC 
output  the  target  altitude  in  discrete  20  ft  increments.  Although  the  WSIC  altitude  output 
remained  constant  until  an  increment  occurred,  the  PDUs  showed  small  variations  in 
altitude.  This  occurred  because  the  WSIC  NIU  dead  reckoned  the  target  altitude  (from 
the  target’s  down  component  of  velocity)  for  the  PDU  data  between  the  WSIC  altitude 
updates.  In  other  words,  the  NIU  attempted  to  correct  the  PDU  data  for  the  fact  that  the 
target  altitude  should  have  actually  been  changing  during  each  altitude  “step.”  When  a 


4-79 


WSIC  target  altitude  update  was  received  at  the  NIU,  the  PDU  data  were  “corrected” 
back  to  the  discrete  values  output  by  the  WSIC. 

-  When  the  WSIC  output  was  replayed,  new  target  entity  state  PDUs  were  created.  As 
Figure  4.2.3.2-11  shows,  the  altitude  variations  in  the  replayed  PDUs  did  not  exactly 
overlay  the  original  PDU  data. 

-  The  variations  in  the  start  of  the  replayed  manual  data  resulted  in  variations  in  the 
engagement  conditions. 

As  Figures  4.2.3.2-10  and  4.2.3.2-1 1  show,  the  replayed  data  tracked  the  manual  run,  except  for 
the  small  altitude  variations.  However,  the  manual  actions  required  for  the  automatic  replay  runs 
resulted  in  the  target  being  at  slightly  different  locations  at  missile  launch,  relative  to  the  manual 
trial.  These  offsets  in  the  target  location  at  launch  are  illustrated  in  Figure  4.2.3.2-12.  The  mean 
and  standard  deviation  of  the  offsets  were  -32.6  +  379.6  ft.  Hence,  there  was  a  small  bias,  but  a 
large  spread  in  the  offset  of  the  target  location  at  launch  for  the  automatic  replay  runs  relative  to 
the  manual  run. 

Another  problem  was  found  with  the  replayed  target  PDU  data.  The  target  PDUs  from  the  replay 
runs  had  repeating  PDUs,  whereas  the  manual  data  did  not.  In  other  words,  during  the  replay, 
several  identical  PDUs  in  a  row  were  logged.  These  repeating  PDUs  all  had  the  same  PDU  time 
and  the  same  entity  state  data.  As  many  as  12  in  a  row  were  observed  on  some  runs.  Since  the 
SIMLAB  missile  simulation  integrated  the  target  velocity  to  determine  the  target  location,  these 
repeating  PDUs  aggravated  the  target  latitude  divergence  problem  (Section  4. 1.1. 3).  In  other 
words,  the  SIMLAB  target  representation  during  the  automatic  replay  runs  was  north  of  that  from 
the  manual  run  being  replayed.  That  this  is  the  case  is  best  illustrated  by  comparing  the  times-of- 
flight.  The  TOF  of  every  automatic  replay  run  was  longer  than  the  TOF  of  the  manual  run,  even 
though  a  number  of  the  replay  runs  had  shorter  launch  ranges  (Fig.  4.2.3.2-6).  The  mean  and 
standard  deviation  of  the  TOFs  for  the  automatic  replay  runs  was  0.33  ±  0.20  seconds  longer  than 

the  TOF  of  the  manual  run. 

Hence,  the  automatic  replay  runs  did  not  achieve  their  purpose:  perfect  replication  of  the  launch 
conditions  and  perfect  replication  of  the  target  trajectory  each  time.  For  this  reason,  the 
automatic  replay  runs  were  not  executed  on  subsequent  missions. 

Time  Required  for  Automatic  Replay  Runs  .  The  total  time  required  to  execute  34  automatic 
V2  runs  was  4  hours  which  averaged  8.5  runs/hr  (~7  min.  between  runs).  This  included  the  set  up 
time  (~1  hour)  and  the  time  for  five  VI  runs  in  preparation  for  the  V2  runs  (10  min.). 


Cost  of  the  Automatic  Replay  Runs  ,  The  cost  of  the  NAWCWPNS  simulation  facilities  was 
$6K/hr.  Thus,  the  total  cost  of  the  automatic  replay  runs  was  $24K. 
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.2.3.2-1.  Shooter  Velocity  Relative  to  Manual  Trial  #15  Value  (Automatic  Replay  Trials) 
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Figure  4.2.3.2-2.  Shooter  Altitude  Relative  to  Manual  Trial  #15  Value  (Automatic  Replay  Trials) 
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Figure  4.2.3.2-3.  Target  Velocity  Relative  to  Manual  Trial  #15  Value  (Automatic  Replay  Trials) 
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Altitude  Relative  to  Manual  Trial  #15  Value  (Automatic  Replay  Trials) 
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Figure  4.2.3.2-5.  Shooter  and  Target  Altitude  Difference  Relative  to  Manual  Trial  #15  Value  (Automatic  Replay  Trials) 
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Figure  4.2.3.2-6.  Launch  Range  Relative  to  Manual  Trial  #15  Value  (Automatic  Replay  Trials) 


Figure  4.2.3.2-7.  Target  Aspect  Angle  Relative  to  Manual  Trial  #15  Value  (Automatic  Replay  Trials) 


Figure  4.2.3.2-8.  Lead  Angle  Relative  to  Manual  Trial  #15  Value  (Automatic  Replay  Trials) 


Figure  4.23.2-9.  Flare  Initiation  Time  Relative  to  Manual  Trial  #15  Value  (Automatic  Replay  Trials) 


Auto  Trial  #14 
Manual  Trial  #15 


Figure  4.2.3.2-10.  Comparison  of  Target  Trajectory  From  Automatic  Replay  Trial  #14  with  Manual  Trial  #15 
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Figure  4.2.3.2-11.  Comparison  of  Target  Trajectory  From  Automatic  Replay  Trial  #14  with  Manual  Trial  #15 
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Figure  4.2.3.2-12.  Offset  in  Target  Position  at  Missile  Launch  Relative  to  Manual  Trial  #15  Value  (Automatic  Replay  Trials) 


4.2.4  Parametric  Study  Summary 

The  manual  method  for  replicating  a  given  profile  resulted  in  very  good  run-to-run  reproducibility 
of  the  launch  conditions.  However,  the  automatic  replay  method  was  unable  to  precisely  replicate 
a  given  scenario.  Hence,  the  manual  method  can  effectively  support  parametric  studies,  but  the 
automatic  replay  method,  as  implemented  in  the  Mission  Rehearsal,  cannot. 

In  order  for  the  automatic  replay  method  to  be  effective,  the  manual  actions  required  in  the  LSP 
trials  need  to  be  replaced  with  automatic  procedures.  This  would  have  required  significant 
modifications  of  the  NAWCWPNS  simulators  and  was  beyond  the  scope  of  the  LSP.  However, 
future  implementations  of  this  concept  may  be  able  to  overcome  this  limitation. 

4.3  Test  Objective  3:  Assess  effect  of  latency  on  validity  of  test  results 

This  objective  is  to  evaluate  the  effects  of  latency  on  test  results.  An  issue  here  is  that  when  there 
is  significant  latency,  the  different  simulation  nodes  experience  a  “different”  engagement. 
Although  the  missile  simulation  may  continue  to  successfully  intercept  the  target  in  the 
engagement  presented  to  it  at  the  SIMLAB  node,  this  will  not  be  exactly  the  same  engagement  as 
that  experienced  by  either  the  target  or  the  shooter.  In  other  words,  with  “too  much”  latency,  the 
planned  engagement  cannot  be  executed.  Further,  “too  much”  latency  will  invalidate  the  results 
of  a  reactive  scenario  (missile  and  target  are  reacting  to  each  other). 

4.3.1  Latency  Study  Test  Method 

The  Latency  Study  Mission  was  designed  to  accomplish  this  test  objective  and  was  to  utilize  a 
reactive  scenario  in  which  the  target  reacted  to  the  missile  from  cues  generated  by  the  missile 
warning  system  model.  Note  that  this  would  not  be  the  V&V  scenario,  LPN-15  (which  did  not 
involve  the  missile  warning  system  model).  Rather,  a  scenario  was  to  be  selected  from  the 
planned  Parametric  Mission  in  which  the  warning  system/CM  clearly  affected  the  missile 
performance. 

One  flight  profile  was  planned  for  the  Latency  Study  Mission:  the  reactive  profile  selected  from 
the  Parametric  Study  Mission.  The  latency  study  was  to  be  performed  by  first  replicating  this 
profile  for  the  baseline  minimum  latency  case  (inherent  system/network  latency  only)  and  noting 
the  engagement  results  for  each  node.  Next  the  profile  was  to  be  repeated  with  an  adjustable  time 
delay  at  the  WSIC  node  only  and  again  the  engagement  results  for  each  node  were  to  be  noted. 
The  delay  was  to  be  between  the  network  and  the  WSIC  NIU  and  was  to  affect  both  incoming 
and  outgoing  PDUs.  The  profile  was  to  be  repeated  with  additional  increments  of  time  delay  at 
the  WSIC  node. 

Features  of  this  mission  were  to  be  as  follows. 

The  time  delays  were  to  be  introduced  only  at  the  WSIC  node.  The  link  between  the 
WSSF  and  the  SIMLAB  was  not  to  be  affected.  The  reason  for  not  adding  time  delays 
between  the  WSSF  and  SIMLAB  was  that  there  were  two  links  between  these  facilities 
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(one  for  PDU  data  and  one  for  SMS  data),  delays  would  have  to  be  added  to  both  links 
simultaneously,  and  significant  delays  could  not  be  added  to  the  SMS  link  without 
affecting  the  proper  operation  of  the  SMS  system.  Adding  delays  only  at  the  WSIC  node 
would  have  simulated  the  situation  of  moving  the  target  node  to  a  geographically  remote 
location,  while  keeping  the  shooter  and  missileco-located. 

-  The  values  of  time  delay  was  to  be  set  in  a  computer  in  the  WSIC  which  was  located 
between  the  Cisco  router  and  the  rest  of  the  WSIC  network.  This  was  the  first  processing 
point  in  the  WSIC  encountered  by  incoming  PDUs  and  the  last  processing  point 
encountered  by  outgoing  PDUs.  The  computer  was  to  buffer  the  PDUs  until  the  preset 
delay  time  had  elapsed.  At  that  time,  the  computer  was  to  transmit  the  PDUs.  The  actual 
simulation-computer-to-simulation-computer  latency  was  to  be  measured  before  each  run 
by  generating  a  fire  flare  signal  in  the  WSIC  (with  generation  time  stamp)  and  recording 
its  input  into  another  simulation  computer  (with  input  time  stamp).  Latency  between  the 
WSIC  and  the  BMIC/TCAC  was  to  determined  from  the  time  stamp  when  the  Fire  (Flare) 
PDU  was  recorded  in  the  PDU  logger  located  in  the  BMIC/TCAC. 

-  Initial  increments  of  time  delay  to  be  implemented  were  to  be  determined  after  analysis  of 
the  V&V  and  Parametric  Mission  results.  The  increments  were  to  be  adjusted  after  quick- 
look  analysis  of  the  initial  results,  in  order  to  better  identify  the  onset  of  significant  latency 
effects. 

As  the  time  delay  was  increased,  the  engagement  viewed  at  each  of  the  four  nodes  (WSSF, 
SIMLAB,  WSIC,  TCAC)  was  expected  to  increasing  differ  in  terms  of  launch  range,  flare 
release/target  maneuver  time,  and  miss  distance. 

4.3.2  Latency  Study  Analysis  Method 

Runs  of  the  selected  profile  were  to  be  evaluated  to  determine  the  range  of  acceptable  initial 
launch  conditions  and  target  trajectories/profiles.  The  evaluation  criterion  was  to  be  that  these 
ranges  of  conditions/parameters  for  the  shooter  and  target  result  in  acceptable  ranges  of  variations 
in  the  missile  trajectory/profiles  (based  on  the  judgment  of  AIM-9  expert).  The  result  was  to  be 
acceptable  ranges  for  the  following: 

Shooter  initial  conditions,  as  measured  in  the  WSSF 

-  Launch  range,  as  measured  in  the  WSSF 

-  Target  initial  conditions,  as  measured  in  the  WSlC 

-  Flare  dispense  time  relative  to  missile  warning  cue,  as  measured  in  the  WSIC 

-  Target  evasive  maneuver  time  and  profile  upon  missile  warning  cue,  as  measured  in  the 
WSIC 

-  The  terms  “as  measured  in  the  WSSF”  or  “as  measured  in  the  WSIC”  mean  that  these 
parameters  were  evaluated  from  data  recorded  and  time  stamped  on  the  simulation 
computer  of  the  corresponding  facility. 
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These  acceptable  ranges  were  to  form  screening  criteria  for  subsequent  runs  involving  additional 
time  delays.  Runs  meeting  the  criteria  were  to  be  analyzed  for  latency  effects,  and  runs  not 
meeting  the  criteria  were  not  to  be  further  analyzed. 

The  runs  meeting  the  screening  criteria  were  to  be  independently  analyzed  at  each  facility  (WSSF, 
SIMLAB,  WSIC,  and  TCAC)  for  each  time  delay  increment.  Using  data  recorded  at  each  facility, 
the  following  were  to  be  determined: 

Launch  Range .  This  was  computed  from  the  difference  in  the  positions  of  the  shooter  and 
the  target  when  either  the  missile  launch  event  indicator  was  recorded  in  the  simulation 
computer  (for  simulation  facilities)  or  when  the  Fire  (Missile)  PDU  was  recorded  in  the 
TCAC  PDU  logger.  The  indicators  used  for  missile  launch  differed  for  the  various 
simulations  and  will  be  discussed  with  the  analysis  results. 

.  Target  Aspect  at  Launch  .  This  was  computed  from  the  difference  in  the  attitudes  of  the 
shooter  and  the  target  when  either  the  missile  fire  event  indicator  was  recorded  in 

the  simulation  computer  (for  simulation  facilities)  or  when  the  Fire  (Missile)  PDU  was 
recorded  in  the  TCAC  PDU  logger.  The  aspect  angle  was  computed  as  follows: 

—  The  shooter  and  target  locations  were  projected  onto  the  local  tangent  plane  (i.e.,  the 
local  horizontal  plane  or  the  “God’s-eye”  view). 

—  The  direction  of  th  e  line-of-sight  (LOS)  vector  from  the  shooter  to  the  target  in  the 
local  tangent  plane, Ira,  (see  Fig.  4.3.2-1)  was  computed  using: 


where  Alon  =  longitude  of  target  -  longitude  of  shooter 
Alat  =  latitude  of  target  -  latitude  of  shooter 
latt  =  latitude  of  target 

-  The  target  aspect  angle,  aA,  was  computed  using  (see  Fig.  4.3 .2- 1 ): 
oA  =  br-SRA 

where  br  =  bearing  of  target  (the  direction  of  the  target’s  velocity  vector,  vT, 
projected  onto  the  local  tangent  plane)  measured  from  north 

-  Initially,  the  target  aspect  angle  was  computed  using  the  target  heading  in  place  of  the 
bearing.  However,  it  was  determined  that  the  shooter’s  radar  uses  the  target  bearing 
in  live  shots.  In  order  to  compare  to  LPN-15  test  data  and  to  the  aspect  angle  from 
the  WSSF  F/A-18  simulator,  the  calculation  for  aspect  angle  was  revised  to  use  target 
bearing  (Eqn.  6).  Note  that  the  target’s  heading  (direction  nose  is  pointed,  as 
projected  on  the  tangent  plane)  does  not  coincide  with  the  target  bearing  when  the 
target  is  turning,  due  to  the  aircraft  angle  of  attack. 


4-95 


(in  tangent  plane) 

Figure  4.3.2-1.  Geometry  for  Calculating  Target  Aspect  Angle,  aA  (positions  and  vT 
projected  into  horizontal  tangent  plane) 


-  Lead  Angle .  This  was  computed  from  the  difference  in  the  heading  of  the  shooter  and  the 
direction  of  the  LOS  vector  when  either  the  missile  fire  event  indicator  was  recorded  in 
the  simulation  computer  (for  simulation  facilities)  or  when  the  Fire  (Missile)  PDU  was 
recorded  in  the  TCAC  PDU  logger.  The  lead  angle  was  computed  as  follows: 

-  The  shooter  and  target  locations  were  projected  onto  the  local  tangent  plane  (i.e.,  the 
local  horizontal  plane  or  the  “God’s-eye”  view). 

-  The  direction  of  the  LOS  vector  from  the  shooter  to  the  target  in  the  local  tangent 
plane,  Era,  (see  Fig.  4.3.2-1)  was  computed  using  Equation  5. 

-  The  lead  angle,  y,  was  computed  using  (see  Fig.  4.3.2-2): 

V|/  =  hs  -  2ra  (^) 

where  hs  =  heading  of  shooter  (the  direction  of  the  sh  ooter’s  nose,  projected 
onto  the  local  tangent  plane)  measured  from  north 

-  Shooter  Altitude .  This  was  determined  by  the  shooter  altitude  when  either  the  missile  fire 
event  indicator  was  recorded  in  the  simulation  computer  (for  simulation  facilities)  or  when 
the  Fire  (Missile)  PDU  was  recorded  in  the  TCAC  PDU  logger. 

-  Time  of  Launch.  This  was  determined  by  the  time  at  which  either  the  missile  fire  event 
indicator  was  recorded  in  the  simulation  computer  (for  simulation  facilities)  or  when  the 
Fire  (Missile)  PDU  was  recorded  in  the  TCAC  PDU  logger. 
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Figure  4.3.2-2.  Geometry  for  Calculating  Lead  Angle, \|/  (positions  projected  into 

horizontal  tangent  plane) 


.  Time  of  Flare  Release  .  This  was  to  determined  by  the  time  at  which  either  the  flare  fire 
event  indicator  was  recorded  in  the  simulation  computer  (for  simulation  facilities)  or  when 
the  Fire  (Flare)  PDU  was  recorded  in  the  TCAC  PDU  logger. 

Time  of  Evasive  Maneuver  .  This  was  to  be  determined  by  the  time  at  which  the  target 
acceleration  began  to  increase  from  the  baseline  value  (3.5  g)  to  the  evasive  maneuver 
value  (7+  g). 

.  Time  of  Missile  Detonation  .  This  was  to  be  determined  by  the  time  at  which  either  the 
missile  detonation  event  indicator  was  recorded  in  the  simulation  computer  (for  simulation 
facilities)  or  when  the  Detonation  PDU  was  recorded  in  the  TCAC  PDU  logger. 

-  Miss  Distance .  This  was  to  be  computed  from  the  difference  in  the  positions  of  the  missile 
and  the  target  when  either  the  missile  detonation  event  indicator  was  recorded  in  the 
simulation  computer  (for  simulation  facilities)  or  when  the  Detonation  PDU  was  recorded 
in  the  TCAC  PDU  logger. 

The  average  values  for  each  of  the  above  parameters,  as  determined  at  each  facility,  were  to  be 
computed  for  each  profile  (i.e.,  for  each  value  of  time  delay). 

The  average  parameter  values  for  the  SIMLAB  stand-alone  profile  (Ll-S)  were  to  be  taken  as  the 
baseline  values. 

For  each  profile  (each  time  delay),  the  difference  between  the  average  value  of  each  parameter  (as 
determined  at  each  of  the  four  facilities)  and  the  baseline  value  was  to  be  computed. 

The  actual  simulation-computer-to-simulation-computer  latency  was  to  have  been  measured  for 
each  profile  (time  delay  setting)  as  follows: 

-  A  fire  flare  signal  was  to  be  manually  generated  in  the  WSIC.  This  signal  was  to  be 
recorded  in  the  WSIC  simulation  computer,  along  with  the  associated  time  stamp  for  time 
of  generation. 
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-  When  the  fire  flare  signal  was  received  at  the  other  simulation  computers,  it  was  to  be 
recorded  there,  along  with  a  time  stamp  for  the  time  of  input  into  the  receiving  computer. 

-  When  the  Fire  (Flare)  PDU  created  at  the  WSIC  was  received  at  the  BMIC/TCAC,  it  was 
to  be  recorded  on  the  PDU  logger  located  there,  along  with  a  time  stamp  for  the  time  of 
receipt. 

-  The  latency  between  each  receiving  simulation  computer  or  the  BMIC/TCAC  and  the 
WSIC  simulation  computer  was  to  be  computed  as  the  difference  between  the  time  of 
receipt  and  the  time  of  generation  at  the  WSIC. 

This  procedure  was  to  be  repeated  a  number  of  times  bef  ore  the  actual  trials  of  the  profile 
began.  Latency  results  were  to  be  analyzed  to  determine  the  following  latency  statistics: 
relative  frequency  histogram,  mean,  median,  mode,  and  variance. 

4.3.3  Latency  Study  Results 

The  Latency  Study  Mission  was  not  executed.  Schedule  slips  caused  by  problems  with  V&V  of 
the  LSP  architecture  resulted  in  the  Parametric  Study  Mission  being  executed  during  the  time 
frame  originally  planned  for  the  Latency  Study  Mission.  In  addition,  an  abundance  of  latency  data 
was  collected  in  the  other  missions,  and  the  Latency  Studies  Mission  was  judged  to  be  redundant. 
Hence,  a  mission  was  not  performed  in  which  latency  was  treated  as  a  controlled  variable. 
Instead,  latency  data  from  the  other  missions  were  used  to  analyze  this  test  objective. 

4.3.3.1  Latency  Characteristics 

First,  the  characteristics  of  latency  were  determined.  The  method  for  computing  latency  discussed 
above  was  to  use  the  Fire  (Flare)  PDU  created  at  the  WSIC  and  assumed  a  constant  latency  value 
between  the  WSIC  and  the  other  nodes.  However,  no  runs  were  completed  in  which  a  Fire 
(Flare)  PDU  was  created  at  the  WSIC.  Further,  the  latency  values  between  each  pair  of  nodes 
were  found  to  vary  significantly  during  a  run. 

Instead,  latency  values  were  calculated  throughout  a  run  from  the  differences  in  time  stamps  for 
the  same  set  of  entity  state  data  logged  at  various  locations.  Latency  was  computed  for  each  set 
of  entity  state  data  transmitted  between  two  logging  locations  dining  a  run,  as  follows: 

-  When  latency  was  to  be  determined  between  a  creating  simulation  and  a  PDU  logger,  the 
values  of  the  entity  state  data  were  used  to  match  a  simulation  frame  with  a  PDU,  and  the 
latency  was  computed  from  the  difference  between  the  PDU  log  time  (when  the  PDU  was 
logged)  and  the  PDU  time  (when  the  data  originated  in  the  creating  simulation). 

-  When  latency  was  to  be  determined  between  PDU  loggers,  the  PDU  time  was  used  to 
match  the  PDUs,  and  the  latency  was  computed  from  the  difference  between  the  log  times. 

-  When  latency  was  to  be  determined  between  a  PDU  logger  and  a  receiving  simulation,  the 
values  of  the  entity  state  data  were  used  to  match  a  PDU  with  a  simulation  frame,  and  the 
latency  was  computed  from  the  difference  between  the  time  stamp  of  the  simulation  frame 
and  the  PDU  log  time. 
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-  When  latency  was  to  be  determined  between  simulations,  the  values  of  the  entity  state  data 
were  used  to  match  simulation  frames,  and  die  latency  was  computed  from  the  difference 
between  the  time  stamps  of  the  matching  simulation  frames. 

The  latency  between  various  pairs  of  logging  locations  was  characterized  by  its  time  variation, 
frequency  distribution,  mean  value,  standard  deviation,  minimum  value,  and  maximum  value. 

Mission  Rehearsal  Latencies 

Latencies  could  not  be  determined  from  the  first  day  of  testing  (8/27/96)  due  to  a  problem  with 
the  logging  time  stamp.  The  time  stamp  on  the  STRICOM  PDU  loggers  was  off  from  the  IRIG 
time  used  by  the  NAWCWPNS  simulators  by  about  15  sec.  This  problem  was  fixed  for  the 
second  day  (8/28/96)  of  the  Mission  Rehearsal.  Tables  4.3.3.1-1  and  4.3.3.1-2  present  samplings 
of  latency  data  from  the  second  day,  and  Figures  4.3.3. 1-1  through  4.3.3. 1-5  show  latency  as  a 
function  of  PDU  time  (the  PDU  time  uniquely  identifies  an  entity  state  PDU)  for  several  entity/run 
combinations. 


Table  4.3.3.1-1.  Latencies  (in  msec)  of  Target  Entity  State  Data  (8/28/96) 


Run  # 

WSIC  Sim  to  WSIC  Logger 

WSIC  Sim  to  S 

1MLAB  1 

Logger 

Mean 

Std 

Dev 

Min 

Max 

Mean 

Std 

Dev 

Min 

Max 

Mean 

1 

276.5 

140.2 

26 

528 

316.3 

126.2 

80 

554 

39.8 

10 

275.5 

139.8 

40 

512 

314.9 

126.1 

87 

543 

39.4 

48 

271.1 

140.3 

32 

576 

311.3 

126.2 

92 

599 

40.2 

51 

266.9 

140.1 

33 

551 

306.1 

126.1 

92 

577 

39.2 

Table  4.3.3.1-2.  Latencies  (in  msec)  of  Shooter  and  Missile  Entity  State  Data  (8/28/96) 


Run  # 

Shooter  Entity  State  Data  (before  launch) 
WS  SF  Sim  to  SIMLAB  Logger 

Missile  Entity  State  Data 

SIMLAB  Sim  to  SIMLAB  Logger 

Mean 

Std  Dev 

Min 

Max 

Mean 

Std  Dev 

Min 

Max 

6 

24.6 

11.3 

6 

63 

677.4 

129.2 

404 

1025 

8 

37.0 

26.8 

13 

166 

548.4" 

112.0 

371 

680 

46 

34.7 

23.8 

6 

137 

593.4 

98.6 

535 

915 

51 

67.7 

120.4 

6 

977 

51.1 

24.5 

-10 

148 

4-99 
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Figure  4.3.3.1-1.  Latency  of  Target  Entity  State  Data  Between  WSIC  Simulation  and  SIMLAB  PDU  Logger  (Run  #51  on 

8/28/96) 
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Figure  4.3.3.1-2.  Latency  of  Shooter  Entity  State  Data  Between  WSSF  Simulation  and  SIMLAB  PDU  Logger  (Run  #51  on 
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Figure  4.3.3.1-3.  Latency  of  Shooter  Entity  State  Data  (before  launch)  Between  WSSF  Simulation  and  SIMLAB  PDU  Logger 

(Run  #46  on  8/28/96) 


(oasui)  Aouajen 


Figure  4.3.3.1-4.  Latency  of  Missile  Entity  State  Data  (before  launch)  Between  SIMLAB  Simulation  and  SIMLAB  PDU 

Logger  (Run  #46  on  8/28/96) 
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Figure  4.3.3.1-5.  Latency  of  Missile  Entity  State  Data  Between  SIMLAB  Simulation  and  SIMLAB  PDU  Loggei 


The  following  is  noted  from  Tables  4.3.3. 1-1  and  4.3.3. 1-2  and  Figures  4.3.3. 1-1  through  4.3.3. 1- 
5: 


-  Target  Entity  State  Data  Latencies 

-  The  mean  latencies  of  the  target  entity  state  data  were  consistent  run-to-run,  but  large. 

-  However,  during  a  single  run,  the  range  of  lat  encies  was  relatively  large  (~500  ms). 
As  Figure  4.3. 3. 1-1  shows,  the  target  entity  state  PDUs  were  received  at  the  SIMLAB 
in  groups  of  12-13,  resulting  in  the  large  latency  range  (the  range  corresponds  to  each 
group).  Note  that  the  latency  versus  time  behavior  in  this  figure  is  quite  repeatable. 

-  As  Table  4.3.3.1-1  shows,  most  of  the  latency  contribution  was  between  the  WSIC 
simulation  and  the  WSIC  PDU  logger.  The  transmission  latency  between  the  WSIC 
and  the  SIMLAB  was  small  by  comparison. 

-  Shooter  Entity  State  Data  Latencies 

--  The  mean  latencies  of  the  shooter  entity  state  data  were  relatively  small  and  consistent 
run-to-run,  with  the  exception  of  Run  #5 1 . 

-  As  Figure  4.3.3. 1-2  shows,  the  latencies  of  the  shooter  data  on  Run  #51  became 
uncharacteristically  large  near  the  launch  time  (note  that  the  maximum  latency  on  this 
run  was  much  larger  than  on  other  runs,  but  the  minimum  latency  was  about  the 
same). 

-  Note  from  Table  4.3.3. 1-2  that  the  maximum  latencies  on  a  run  were  typically  many 
(4-7)  standard  deviations  above  the  mean  value.  On  the  other  hand,  minimum  latency 
was  usually  within  one  standard  deviation  of  the  mean.  Figure  4.3.3. 1-3  shows 
another  example  of  the  shooter  entity  state  data  latency  versus  time  in  which  the  high 
latency  values  are  seen  to  occur  over  a  relatively  short  time,  but  not  at  the  same  time 
relative  to  launch  as  in  Figure  4.3. 3. 1-2.  It  is  hypothesized  that  large  latencies  in  the 
shooter  entity  state  data  may  have  been  caused  by  heavy  computation  loads  in  the 
WSSF  simulation  computer  as  the  shooter  prepared  to  launch  the  missile. 

-  As  Figure  4.3.3. 1-3  shows,  there  was  also  some  grouping  of  shooter  entity  state 
PDUs,  but  the  groups  were  smaller  than  for  the  target  entity  state  PDUs  (3-4  per 
group  versus  12-13). 

-  The  shooter  entity  state  PDUs  were  observed  to  frequently  “repeat,”  i.e.,  several 
PDUs  were  observed  which  had  the  same  PDU  time  and  the  same  entity  state  data,  but 
different  log  times  (groups  of  repeating  PDUs  as  large  as  eight  were  observed).  The 
reason  for  this  was  unclear,  but  may  have  been  due  to  the  WSSF  NIU  creating  PDUs 
at  a  preset  time  before  updated  entity  state  data  had  been  received  from  the  WSSF 
simulation.  Since  repeating  PDUs  did  not  represent  a  valid  update  in  the  shooter’s 
entity  state,  these  were  eliminated  from  latency  calculations.  Instead,  repeating  PDUs 
were  treated  as  ADS-induced  errors  and  analyzed  under  Test  Objective  4-2. 

-  Missile  Entity  State  Data  Latencies 

-  The  mean  latencies  of  the  missile  entity  state  data  were  large  and  consistent  run-to- 
run,  with  the  exception  of  Rim  #5 1 . 
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-  Figure  4.3.3. 1-4  shows  the  latency  versus  time  for  one  of  the  high  latency  runs.  Note 
that  there  was  a  relatively  consistent  high  latency  baseline  with  higher  latency  values 
over  a  short  duration  near  the  end  of  the  missile  flyout. 

—  The  latency  versus  time  for  Run  #51  is  shown  in  Figure  4.3.3. 1-5.  This  run  differed 
significantly  from  the  other  runs,  having  a  mean  latency  10%,  or  less,  of  the  means  for 
the  previous  runs.  Note  that  Run  #51  had  negative  latencies  for  a  short  duration 
which  are  physically  impossible.  The  cause  of  these  negative  latencies  may  have  been 
an  error  in  the  time  stamp  applied  during  logging  in  the  STRICOM  logger.  Time 
stamping  errors  may  also  account  for  the  unusually  small  latencies  for  Run  #51.  Since 
negative  latencies  cannot  be  correct,  such  values  were  eliminated  from  latency  analyses 
in  subsequent  missions. 

SUMMARY.  The  mean  latencies  of  the  target  and  missile  entity  state  data  were  large  and  had 
large  variations  during  runs  with  maximum  latencies  up  to  one  second.  The  latencies  of  shooter 
entity  state  data  had  much  smaller  mean  values  by  comparison,  but  had  some  high  latency 
“spikes”  (up  to  ~1  sec).  The  smaller  shooter  mean  latencies  suggested  that  correspondingly  small 
latencies  should  also  be  possible  for  the  target  and  missile  entity  state  data.  Most  of  the  latency 
contribution  was  between  the  simulation  and  the  NIU  where  PDUs  were  created  from  the  raw 
simulation  data.  Transmission  latencies  between  nodes  were  relatively  small. 

V&V  Mission  Latencies 


Latencies  for  the  V&V  Mission  (10/29/96)  runs  were  analyzed  as  follows: 

-  The  latencies  of  the  shooter  entity  state  data  between  the  WSSF  and  the  WSIC  and  of  the 
target  entity  state  data  between  the  WSIC  and  the  WSSF  were  analyzed  prior  to  missile 
launch.  Before  launch,  the  shooter  tracks  the  target,  and  low  latencies  are  needed  if  the 
shooter  and  target  are  to  agree  on  the  launch  conditions.  The  latency  of  the  shooter  data 
going  to  the  WSIC  was  analyzed  to  determine  implications  for  potential  future 
applications  in  which  the  target  and  shooter  are  “dog  fighting”  before  launch. 

-  The  latencies  of  the  target  entity  state  data  between  the  WSIC  and  the  SIMLAB  and  of 
the  missile  entity  state  data  between  the  SIMLAB  and  the  WSIC  were  analyzed  after 
missile  launch  and  during  the  missile  flyout  to  the  target.  The  latency  of  the  target  data 
going  to  the  SIMLAB  was  analyzed  to  determine  how  closely  to  real-time  the  missile  was 
following  the  target.  The  latency  of  the  missile  data  going  to  the  WSIC  was  analyzed  to 
determine  implications  for  potential  future  applications  in  which  the  target  and  missile  are 
interacting  to  each  other  in  a  “closed-loop”  fashion. 

-  Tables  4.3.3. 1-3  through  4.3.3.1-6  present  a  sampling  of  runs  during  the  V&V  Mission, 
and  include  entries  for  the  latency  characteristics  between  the  originating  simulation  and 
the  logger  at  the  originating  simulation  node,  the  latencies  between  the  logger  at  the 
transmitting  NIU  and  the  logger  at  the  receiving  NIU,  and  the  net  latencies  between  the 
originating  simulation  and  the  receiving  NIU.  Latencies  into  the  receiving  simulation  were 
only  analyzed  at  launch  for  this  mission  and  will  be  discussed  in  Section  4. 3. 3.2. 

-  Figures  4.3.3. 1-6  through  4.3.3.1-10  show  latency  as  a  function  of  PDU  time  for  several 
entity/run  combinations. 


4-106 


Table  4.3.3.1-3.  Latencies  (in  msec)  for  Target  Entity  State  Data  (before  launch)  From  WSIC  to  WSSF  (10/29/96) 

““  WSIC  Sim  to  WSIC  Logger  I  WSIC  Logger  to  WSSF  Logger  I  WSIC  Sim  to  WSSF  Logge 

Run  #  Mean  Std  Dev  Min  Max  Mean  Std  Dev  Min  Max  Mean  Std  Dev  Min  _ I 

3  23.5  14.6  3  52  17.5  4.7  1  40  41.0  14.3  20 


Table  4.3.3.1-5.  Latencies  (in  msec)  for  Target  Entity  State  Data  (during  missile  flyout)  From  WSIC  to  SIMLAB  (10/29/96) 

WSIC  Sim  to  WSIC  Logger _ WSIC  Logger  to  SIMLAB  Logger _ WSIC  Sim  to  SIMLAB  Logger 

Run  #  Mean  Std  Dev  Min  Max  Mean  Std  Dev  Min  Max  Mean  Std  Dev  Min  Max 
3  23.1  14.2  3  45  16.2  2.7  13  30  39.3  14.2  18  68 


50  - 


igure  4.3.3.1-6.  Latency  of  Target  Entity  State  Data  (before  launch)  Between  WSIC  Simulation  and  WSIC  PDU  Logger 

(Run  #19  on  10/29//96) 
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Figure  4.3.3.1-7.  Latency  of  Shooter  Entity  State  Data  (before  launch)  Between  WSSF  Simulation  and  WSSF  PDU  Logger 

(Run  #3  on  10/29/96) 


1000  1 


Figure  4.3.3.1-8.  Latency  of  Shooter  Entity  State  Data  (before  launch)  Between  WSSF  Simulation  and  WSSF  PDU  Logger 

(Run  #19  on  10/29/96) 
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Figure  4.3.3.1-9.  Latency  of  Missile  Entity  State  Data  Between  SIMLAB  Simulation  and  SIMLAB 

10/29/96) 
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Figure  4.3.3.1-10.  Latency  of  Missile  Entity  State  Data  Between  SIMLAB  Simulation  and  SIMLAB  PDU  Logger  (Run 

10/29/96) 


The  following  is  noted  from  Tables  4. 3. 3. 1-3  through  4.3.3. 1-6  and  Figures  4.3. 3. 1-6  through 
4.3.3.1-10: 

-  Target  Entity  State  Data  Latencies 

«  The  mean  latencies  of  the  target  entity  state  data  were  small  and  consistent  run-to-run 
(Tables  4.3.3. 1-3  and  4.3.3.1-5).  The  mean  latencies  between  the  WSIC  simulation 
and  the  WSIC  PDU  logger  were  -10%  of  the  values  from  the  Mission  Rehearsal. 
Also,  the  mean  latencies  between  the  WSIC  PDU  logger  and  the  SIMLAB  PDU 
logger  were  about  half  of  the  values  from  the  Mission  Rehearsal. 

-  As  Figure  4.3.3.1-6  shows,  there  was  still  grouping  of  the  target  entity  state  PDUs, 
although  the  groups  were  smaller  than  in  the  Mission  Rehearsal  (2-3  per  group  versus 
12-13).  Part  of  the  reason  for  the  smaller  number  per  group  may  have  been  that  the 
target  entity  state  PDUs  were  created  at  a  lower  rate  compared  to  the  Mission 
Rehearsal  (12.5  Hz  versus  25  Hz). 

-  As  Tables  4.3.3. 1-4  and  4.3.3.1-5  show,  the  mean  latency  between  the  WSIC 
simulation  and  the  WSIC  PDU  logger  was  about  the  same  as  the  mean  transmission 
latency  between  PDU  loggers.  Also,  the  mean  latency  between  the  WSIC  logger  and 
the  WSSF  logger  was  about  the  same  as  between  all  other  pairs  of  loggers. 

-  Shooter  Entity  State  Data  Latencies 

—  The  mean  latencies  of  the  shooter  entity  state  data  were  large  and  inconsistent  run-to- 
run  (Table  4.3.3. 1-4).  Note  that  the  mean  latency  values  generally  increased  as  the 
runs  progressed.  The  mean  latencies  from  the  earlier  V&V  Mission  runs  were 
comparable  to  those  from  the  Mission  Rehearsal,  but  those  from  the  later  runs  were 
more  than  ten  times  larger. 

-  Most  of  the  runs  still  exhibited  the  large  latency  “spikes,”  as  in  the  Mission  Rehearsal. 
Again,  these  high  latency  values  occurred  over  relatively  short  durations  (200-400 
msec)  at  various  times  before  missile  launch. 

-  Figures  4.3.3. 1-7  and  4.3.3.1-8  compare  shooter  entity  state  data  latencies  for  an  early 
low-mean-latency  run  with  those  for  a  later  high-mean-latency  run.  Note  that  there 
was  a  relatively  consistent  latency  baseline  on  each  run  which  increased.  However,  the 
value  of  the  latency  “spikes”  were  about  the  same  for  the  two  runs. 

Missile  Entity  State  Data  Latencies 

--  The  mean  latencies  of  the  missile  entity  state  data  were  also  large  and  inconsistent  run- 
to-run  (Table  4.3.3.1-6).  However,  unlike  the  shooter  entity  state  data  latencies,  the 
mean  latencies  of  the  missile  data  did  not  show  a  progressive  increase.  The  range  of 
mean  latencies  was  somewhat  smaller  than  that  for  the  Mission  Rehearsal. 

-  Figures  4.3.3. 1-9  and  4.3.3.1-10  compare  missile  entity  state  data  latencies  for  a  low- 
mean-latency  run  with  those  for  a  high-mean-latency  run.  Note  that  there  was  a 
relatively  consistent  latency  baseline  on  each  run  which  increased.  Also,  note  the 
latency  “spikes.” 

-  Figure  4.3.3.1-10  shows  that  th  e  missile  entity  state  PDUs  were  “bunching”  on  some 
of  the  runs  with  10-12  PDUs  per  group. 
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SUMMARY.  The  mean  latencies  of  the  target  entity  state  data  were  much  smaller  than  in  the 
Mission  Rehearsal,  while  those  for  the  shooter  were  much  larger.  The  cause(s)  of  these  changes 
was  not  obvious,  since  there  were  changes  in  both  the  code  used  to  program  the  NIUs  and  in  the 
N1U  parameters.  The  shooter  entity  state  data  still  exhibited  high  latency  “spikes”  (up  to  ~1  sec). 
The  shooter  and  missile  entity  state  data  both  had  large  variations  in  the  mean  latency  run-to-run. 
These  variations  achieved  in  an  uncontrolled  fashion  what  the  Latency  Mission  test  method  was  to 
achieve  in  a  controlled  fashion,  so  that  data  from  this  mission  could  be  used  to  evaluate  the  effects 
of  latency  on  test  results.  Transmission  latencies  between  the  simulation  nodes  were  all  relatively 
small,  and  the  mean  value  was  about  20  msec  for  all  the  node  pairs. 

Parametric  Study  Mission  Latencies 

Subsequent  to  the  V&V  Mission,  it  was  determined  that  resetting  the  NIUs  at  each  node  resulted 
in  lower  latencies  between  the  simulation  at  the  node  and  the  DIS  PDU  logger.  Apparently,  the 
NIU  performance  degraded  as  the  number  of  runs  increased,  as  Table  4.3.3.1-4  suggests.  Also, 
the  improvement  in  latencies  between  the  SIMLAB  simulation  and  the  SIMLAB  DIS  PDU  logger 
(e.g..  Run  #10  in  Table  4.3.3. 1-6)  seemed  to  be  related  to  resetting  the  SIMLAB  NIU.  Hence,  in 
the  Parametric  Study  Mission,  the  WSSF  and  SIMLAB  NIUs  were  reset  before  each  run  (the 
WSIC  NIU  did  not  require  resetting). 

Also,  the  latencies  seemed  to  be  affected  by  the  choice  of  NIU  parameters.  During  the  linked 
laboratory  time  on  1 1/12/96,  the  best  configurations  for  the  NIUs  were  determined  to  be  as  given 
in  Table  4.3. 3. 1-7.  These  settings  were  used  in  the  Parametric  Study  Mission. 


Table  4.3.3.1-7.  NIU  Configurations  for  the  Parametric  Study  Mission 


NIU  Parameter 

WSIC 

WSSF 

SIMLAB 

Timeout 

■ETlMimCTriil  i  ftl 

Dead  Reckoning 

No 

Yes 

Yes 

Iteration  Rate  (NIU  Input  Rate) 

40  Hz 

40  Hz 

40  Hz 

20  Hz 

20  Hz 

20  Hz 

Latencies  for  the  Parametric  Study  Mission  runs  were  analyzed  as  for  the  V&V  Mission. 

-  Tables  4.3.3. 1-8  through  4.3.3.1-11  give  latency  characteristics  for  all  complete  V2  runs, 
and  include  entries  for  the  latencies  between  die  originating  simulation  and  the  logger  at 
the  originating  simulation  node,  the  latencies  between  the  logger  at  the  transmitting  NIU 
and  the  logger  at  the  receiving  NIU,  and  the  net  latencies  between  the  originating 
simulation  and  the  receiving  NIU. 

Latencies  into  the  receiving  simulation  were  determined  for  target  data  into  the  WSSF  and 
the  SIMLAB  simulations  by  matching  entity  state  data  values  in  die  PDUs  with  those  in 
the  simulation  frames.  Results  are  in  Tables  4.3.3.1-12  and  4.3.3.1-13. 

-  Figures  4.3.3. 1-1 1  through  4.3.3.1-24  show  latency  as  a  function  of  PDU  time  and  latency 
frequency  histograms  for  several  entity/run  combinations. 
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Table  4.3.3.1-8.  Latencies  (in  msec)  for  Target  Entity  State  Data  (before  launch)  From  WSIC  to  WSSF  (11/19/96) 

WSIC  Sim  to  WSIC  Logger  1  WSIC  Logger  to  WSSF  Logger  1  WSIC  Sim  to  WSSF  Logger 
Run  #  Mean  Std  Dev  Min  Max  Mean  Std  Dev  Min  Max  Mean  Std  Dev  Min  Max 
9  11.1  9.1  3  105  16.9  4.5  2  42  28.0  9.8  19  122 


^  p:  M  2  o 

^o  2  ^  2  ^ 


rH  oo  (N  rn  (N 
h  oo 

N  (N  (N  n  M 


s  B  e  S  s 


in  co  oo  cq 
id  ©  on  Tf 
(N  (N  ^ 


o  a  >  a 

»  a  s  a 

2  <L>  Q 

§  s  £  s 

<  *8  55  <8 


^  oo  oo  h 

5  ^  n  in  h  ^ 

1-H  t— (  i—H  r— H  »— I  ("Si 


Ol  OO  VO  00  h  00 
(N  (N  (N  N  ^ 


cq  cq  rq  cn  ©  vo 

oo  vd  d  oo 


§  o  oo  vq  a\  h  oo 
in  (N  o  in  © 

Jg  r-  i>  rr  r-  r-  10 


^  CO  o 

IB  |  £  2  ^ 


co  oo  m  h 


in  cq  in  cn  ©  vo 
t-H  ud  vd  ^  d  oo 


M  OO  VO 


O 


S  M M 


cd  m  vo  vo 


•H  M  (N  (N  rf  00 
^  U  w  H  H  Tf 


(N  cj  q  q  q 
ir!  O)  oo  h  h 


g  o  p  q  <n  q  <n 

V  m  *-h  O  On  00 
in  in  on  in  in  tj- 


W  .  CQ 

2  u  Q  o 

<  ‘g  55  ‘g 


Table  4.3.3.1-10.  Latencies  (in  msec)  for  Target  Entity  State  Data  (during  missile  flyout)  From  WSIC  to  SIMLAB  (11/19/96) 

_ WSIC  Sim  to  WSIC  Logger _ WSIC  Logger  to  SIMLAB  Logger _ WSIC  Sim  to  SIMLAB  Logger 

Run#  Mean  Std  Dev  Min  Max  Mean  Std  Dev  Min  Max  Mean  Std  Dev  Min  Max 
9  9.8  4.8  3  20  13.1  3.1  7  37  23.6  5.4  16  55 
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Table  4.3.3.1-12.  Latencies  (in  msec)  for  Target  Entity  State  Data  (before  launch)  From 

WSIC  into  WSSF  Simulation  (11/19/96) 


Run  # 

WSSF  Logger  to  WSSF  Sim 

WSIC  Sim  to  WSSF  Sim  | 

Mean 

Std  Dev 

Min 

Max 

Mean 

Std  Dev 

Min 

Max 

9 

20.6 

25.3 

1 

162 

47.6 

26.5 

30 

190 

10 

38.9 

14.3 

27 

92 

66.4 

14.5 

60 

124 

12 

51.2 

51.5 

1 

347 

77.7 

50.0 

21 

369 

18 

34.9 

12.0 

18 

98 

63.5 

12.8 

43 

120 

19 

33.9 

11.2 

14 

86 

63.9 

9.7 

49 

111 

22 

49.0 

28.4 

3 

187 

78.1 

29.6 

27 

220  j 

Average 
of  Means 

38.1 

23.8 

66.2 

23.9 

Std  Dev  of 
Means 

11.2 

11.2 

Table  4.3.3.1-13.  Latencies  (in  msec)  for  Target  Entity  State  Data  (after  launch)  From 
WSIC  into  SIMLAB  Simulation  (11/19/96) 


Run  # 

SIMLAB  Logger  to  SIMLAB  Sim 

WSIC  Sim  to  SIMLAB  Sim  1 

Mean 

Std  Dev 

Min 

Max 

Mean 

Std  Dev 

Min 

Max 

9 

47.1 

47.2 

12 

328 

70.3 

46.3 

39 

346 

10 

30.5 

30.1 

6 

215 

54.0 

30.3 

25 

235 

12 

53.0 

36.0 

7 

227 

76.0 

36.7 

25 

254 

18 

47.5 

51.8 

2 

362 

71.4 

51.8 

25 

384 

19 

52.3 

45.7 

2 

301 

83.8 

49.0 

28 

328 

22 

39.9 

31.4 

3 

210 

64.6 

32.9 

27 

240 

Average 
of  Means 

45.1 

40.4 

70.0 

41.2 

Std  Dev  of 

Means 


8.5 


10.2 
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Figure  4.3.3.1-lla.  Latency  of  Target  Entity  State  Data  (before  launch)  Between  WSIC 
Simulation  and  WSSF  PDU  Logger  (Run  #22  on  11/19/96) 
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Figure  4.3.3.1-llb.  Frequency  Histogram  of  Latency  of  Target  Entity  State  Data  (before 
launch)  Between  WSIC  Simulation  and  WSSF  PDU  Logger  (Run  #22  on  11/19/96) 
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igure  4.3.3.1-12.  Latency  of  Target  Entity  State  Data  (before  launch)  Between  WSIC 
Simulation  and  WSIC  PDU  Logger  (Run  #22  on  11/19/96) 
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Figure  4.3.3.1-13.  Latency  of  Target  Entity  State  Data  (before  launch)  Between  WSIC 
PDU  Logger  and  WSSF  PDU  Logger  (Run  #22  on  1 1/19/96) 
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Figure  4.3.3.1-14a.  Latency  of  Target  Entity  State  Data  (before  launch)  Between  WSIC 
Simulation  and  WSSF  PDU  Logger  (Run  #19  on  11/19/96) 
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igure  4.3.3.1-15.  Latency  of  Target  Entity  State  Data  (before  launch)  Between  WSIC 
Simulation  and  WSIC  PDU  Logger  (Run  #19  on  11/19/96) 


140 

120 

100 

80 

60 

40 

20 

0 

66210000  66212000  66214000  66216000  66218000  66220000  66222000  66224000  66226000 

PDU  Time  (msec  past  midnight) 

Figure  4.3.3.1-16.  Latency  of  Target  Entity  State  Data  (before  launch)  Between  WSIC 
PDU  Logger  and  WSSF  PDU  Logger  (Run  #19  on  1 1/19/96) 
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Figure  4.3.3.1-17a.  Latency  of  Shooter  Entity  State  Data  (before  launch)  Between  WSSF 
Simulation  and  WSIC  PDU  Logger  (Run  #9  on  11/19/96) 


Figure  4.3.3.1-17b.  Frequency  Histogram  of  Latency  of  Shooter  Entity  State  Data  (before 
launch)  Between  WSSF  Simulation  and  WSIC  PDU  Logger  (Run  #9  on  11/19/96) 
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Figure  4.3.3.1-18a.  Latency  of  Shooter  Entity  State  Data  (before  launch)  Between  WSSF 
Simulation  and  WSIC  PDU  Logger  (Run  #12  on  11/19/96) 


Figure  4.3.3.1-18b.  Frequency  Histogram  of  Latency  of  Shooter  Entity  State  Data  (before 
launch)  Between  WSSF  Simulation  and  WSIC  PDU  Logger  (Run  #12  on  11/19/96) 
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igure  4.3.3.1-19.  Latency  of  Shooter  Entity  State  Data  (before  launch)  Between  WSSF 
Simulation  and  WSSF  PDU  Logger  (Run  #12  on  11/19/96) 
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Figure  4.3.3.1-20.  Latency  of  Shooter  Entity  State  Data  (before  launch)  Between  WSSF 
PDU  Logger  and  WSIC  PDU  Logger  (Run  #12  on  11/19/96) 
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Figure  4.3.3.1-21a.  Latency  of  Target  Entity  State  Data  (after  launch)  Between  WSIC 
Simulation  and  SIMLAB  PDU  Logger  (Run  #9  on  1 1/19/96) 


Figure  4.3.3.1-21b.  Frequency  Histogram  of  Latency  of  Target  Entity  State  Data  (after 
launch)  Between  WSIC  Simulation  and  SIMLAB  PDU  Logger  (Run  #9  on  11/19/96) 


4-126 


66225000  66226000  66227000  66228000  66229000  66230000  66231000  66232000  66233000 

PDU  Time  (msec  past  midnight) 

Figure  4.3.3.1-22a.  Latency  of  Target  Entity  State  Data  (after  launch)  Between  WSIC 
Simulation  and  SIMLAB  PDU  Logger  (Run  #19  on  11/19/96) 


Figure  4.3.3.1-22b.  Frequency  Histogram  of  Latency  of  Target  Entity  State  Data  (after 
launch)  Between  WSIC  Simulation  and  SIMLAB  PDU  Logger  (Run  #19  on  11/19/96) 
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Figure  4.3.3.1-23a.  Latency  of  Missile  Entity  State  Data  Between  SIMLAB  Simulation  and 

WSIC  PDU  Logger  (Run  #9  on  1 1/19/96) 


Figure  4.3.3.1-23b.  Frequency  Histogram  of  Latency  of  Missile  Entity  State  Data  Between 
SIMLAB  Simulation  and  WSIC  PDU  Logger  (Run  #9  on  11/19/96) 
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Figure  4.3.3.1-24a.  Latency  of  Missile  Entity  State  Data  Between  SIMLAB  Simulation  and 

WSIC  PDU  Logger  (Run  #12  on  11/19/96) 


Figure  4.3.3.1-24b.  Frequency  Histogram  of  Latency  of  Missile  Entity  State  Data  Between 
SIMLAB  Simulation  and  WSIC  PDU  Logger  (Run  #12  on  11/19/96) 
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The  following  is  noted  from  Tables  4.3.3.1-8  through  4.3.3.1-13  and  Figures  4.3.3.1-11  through 

4.3.3.1-24: 

-  Target  Entity  State  Data  Latencies  (before  launch) 

-  The  mean  latencies  of  the  target  entity  state  data  were  very  small  and  consistent  run- 
to-run  (Table  4.3.3. 1-8).  The  mean  latencies  between  the  WSIC  simulation  and  the 
WSIC  PDU  logger  were  -50%  of  the  values  from  the  V&V  Mission.  The  mean 
latencies  between  the  WSIC  PDU  logger  and  the  WSSF  PDU  logger  were  essentially 
the  same  as  for  the  V&V  Mission. 

_  As  Table  4.3.3.1-8  shows,  the  mean  latency  between  the  WSIC  simulation  and  the 
WSIC  PDU  logger  was  about  half  of  the  mean  transmission  latency  between  PDU 
loggers. 

-  As  Figure  4.3.3.1-11  shows,  there  was  grouping  of  the  target  entity  state  PDUs  into 
two  groups.  This  is  shown  most  clearly  in  the  latency  frequency  histogram  (Fig. 

4.3.3. 1- 1  lb). 

-  Comparing  Figures  4.3.3.1-12  and  4.3.3.1-13  to  Figure  4.3.3.1-lla  shows  that  the 
grouping  was  a  characteristic  of  the  WSIC  simulation/WSIC  NIU  output.  Figure 

4.3.3.1- 12  shows  that  the  target  PDUs  logged  at  the  WSIC  either  had  a  latency  of  -5 
msec  or  -15  msec.  On  the  other  hand,  the  latency  between  the  WSIC  PDU  logger  and 
the  WSSF  PDU  logger  displayed  a  single  group  centered  about  the  mean  latency  (18 
msec). 

-  Figure  4.3.3.1-14  shows  the  target  entity  state  data  latency  on  a  run  in  which  some 
uncharacteristically  large  latency  values  appeared  over  a  relatively  short  duration.  The 
latency  frequency  histogram  (Fig.  4.3.3. l-14b)  shows  that  there  were  two  primary 
latency  groups,  as  before,  along  with  several  high  latency  “spikes”  (latencies 
significantly  exceeding  the  mean-plus-one-standard-deviation  value). 

--  Comparing  Figures  4.  3.3.1-15  and  4.3.3.1-16  to  Figure  4.3.3.1-14a  shows  that  the 
latency  “spikes”  were  associated  with  the  latency  between  the  WSIC  and  the  WSSF 
PDU  loggers.  In  other  words,  the  latency  between  nodes  was  not  always  well 
behaved. 

—  Some  of  the  latency  “spikes”  may  have  been  due  to  delays  in  logging  the  PDUs  in 
the  WSSF  PDU  logger,  rather  than  variations  in  the  transmission  latency.  By 
comparing  PDU  data  logged  by  the  SNAP  loggers  with  that  logged  by  the  DIS 
(STRICOM)  loggers,  it  was  determined  that  delays  of  tens  of  msec  could  occur  in 
logging  the  PDUs  at  the  DIS  PDU  loggers  (copies  of  the  PDU  data  were  buffered 
at  the  loggers  before  the  data  were  time  stamped  and  recorded;  this  did  not  delay 
transmission  of  the  PDUs).  However,  on  average  the  DIS  PDU  logger  agreed 
with  the  SNAP  loggers  to  within  one  msec,  the  resolution  of  the  PDU  creation  and 
logging  time  stamps  (all  logger  latencies  reported  in  this  section  are  for  data 
logged  on  the  DIS  loggers). 

-  The  latency  of  the  target  entity  state  data  from  the  W  SSF  PDU  logger  into  the  WSSF 
simulation  were  determined  by  matching  data  in  the  target  PDUs  with  data  logged  in 
the  WSSF  simulation.  The  results  are  given  in  Table  4.3.3.1-12. 

_ The  entity  state  data  used  for  the  match  were  the  east  velocity  components  of  the 

target.  The  north  or  east  velocity  components  usually  gave  a  clear  match  between 
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PDU  data  and  simulation  data,  whereas  positional  data  did  not.  The  reason  was 
that  the  positional  data  were  dead  reckoned  by  the  WSSF  NIU  between  PDU 
updates,  whereas  the  velocity  data  were  not. 

—  Comparing  Table  4.3.3.1-12  with  Table  4.3.3. 1-8  shows  that  the  mean  latency  of 
the  target  data  between  the  WSSF  PDU  logger  and  the  WSSF  simulation  was 
about  four  times  that  between  the  WSIC  simulation  and  the  WSIC  PDU  logger. 
Note  that  a  number  of  latency  determinations  gave  negative  values  (most  likely  due 
to  time  stamping  errors),  so  that  the  latencies  given  in  Table  4.3.3.1-12  are 
suspected  of  being  too  low. 

...  The  total  mean  latency  between  the  WSIC  simulation  and  t  he  WSSF  simulation 
shows  that  the  WSSF  simulation  was  usually  running  one  PDU  update  time  (50 
msec)  behind  the  WSIC  simulation  on  five  of  the  runs  and  within  one  PDU  update 
time  of  the  WSIC  simulation  on  one  of  the  runs.  However,  there  were  instances 
during  a  run  in  which  an  individual  frame  of  the  WSSF  simulation  was  as  many  as 
seven  PDU  update  times  (350  msec)  behind  the  WSIC  simulation. 

-  The  maximum  values  of  latency  between  the  WSSF  logger  and  the  WSSF 
simulation  resulted  when  an  updated  target  PDU  was  not  received  by  the  WSSF 
NIU  by  the  time  data  was  to  be  logged  by  the  WSSF  simulation  (every  50  msec). 
When  this  occurred,  the  NIU  dead  reckoned  the  positional  data  from  the  last  PDU 
for  the  input  into  the  WSSF  simulation.  However,  the  PDU  velocity  data  were  not 
dead  reckoned.  The  result  was  that  the  target  positional  data  changed  relatively 
smoothly  when  the  PDUs  were  not  being  updated  regularly,  but  the  velocity  data 
did  not.  This  is  an  example  of  an  ADS-induced  error  and  will  be  evaluated  under 
Test  Objective  4-2. 

Shooter  Entity  State  Data  Latencies  (before  launch) 

-  The  mean  latencies  of  the  shooter  entity  state  data  were  small  and  consistent  run-to- 
run  (Table  4.3.3. 1-9).  The  mean  latencies  between  the  WSSF  simulation  and  the 
WSSF  PDU  logger  were  about  the  same  as  those  for  the  lowest  mean  latency  run  from 
die  V&V  Mission  and  did  not  show  the  progressive  growth  seen  in  the  V&V  Mission. 
The  mean  latencies  between  the  WSSF  PDU  logger  and  the  WSIC  PDU  logger  were 
essentially  the  same  as  for  the  V&V  Mission. 

-  As  Table  4.3.3.1-9  shows,  the  mean  latency  between  the  WSSF  simulation  and  the 
WSSF  PDU  logger  was  more  than  twice  the  mean  transmission  latency  between  PDU 
loggers. 

-  As  Figure  4.3.3.1-17  shows,  the  latencies  primarily  fell  into  a  single  group.  However, 
there  were  a  number  of  lower  and  higher  latency  “spikes”  falling  outside  the  group. 

-  Table  4.3.3.1-9  shows  that  Run  #12  had  an  unusually  low  mean  latency.  Figure 
4.3.3.1-18  shows  that  die  latency  on  this  run  began  lower  than  the  mean  values  on 
other  runs  and  then  shifted  to  an  even  lower  value.  This  shift  resulted  in  the  latencies 
being  primarily  in  two  groups  for  this  run  (Fig.  4.3.3.1-1 8b). 

-  Figure  4.3.3.1-18a  also  shows  that  the  latencies  on  Run  #12  exhibited  “spiking”  just 
before  missile  launch.  Comparing  Figures  4.3.3.1-19  and  4.3.3.1-20  to  Figure  4.3.3. 1- 
18a  shows  that  the  “spiking”  was  associated  with  the  latency  between  the  WSSF  and 
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WSIC  PDU  loggers.  Again,  here  is  an  example  of  the  latency  between  nodes  not 
being  well  behaved. 

-  The  mean  latency  of  the  shooter  entity  state  data  from  the  WSIC  PDU  logger  into  the 
WSIC  simulation  was  checked  and  found  to  be  relatively  large  (~100  msec).  The 
shooter  data  were  not  used  by  the  WSIC  simulation;  they  were  only  logged  there. 
These  latency  values  do  not  necessarily  represent  the  latency  of  inputting  the  shooter 
data  into  the  WSIC  simulation. 

Target  Entity  State  Data  Latencies  (after  launch) 

-  The  mean  latencies  of  the  target  entity  state  data  after  launch  (Table  4.3.3.1-10)  were 
essentially  the  same  as  before  the  launch  (Table  4.3.3. 1-8).  The  mean  latencies 
between  the  WSIC  PDU  logger  and  the  SIMLAB  PDU  logger  were  within  2  msec  of 
those  between  the  WSIC  PDU  logger  and  the  WSSF  PDU  logger. 

-  As  Figure  4.3.3.1-21  shows,  the  grouping  of  the  target  entity  state  PDUs  into  two 
groups  continued  after  missile  launch.  This  is  shown  most  clearly  in  the  latency 
frequency  histogram  (Fig.  4.3 .3. 1  -21b).  Note  the  high  latency  “spike.” 

-  Figure  4.3.3.1-22  shows  that  die  target  entity  st  ate  data  exhibited  a  burst  of  high 
latency  “spikes”  on  Run  #19,  much  like  the  burst  before  launch  (Fig.  4.3.3.1-14).  As 
Table  4.3.3.1-10  shows,  this  burst  was  associated  with  the  latency  between  the  WSIC 
and  the  SIMLAB  PDU  loggers,  rather  than  the  latency  between  the  WSIC  simulation 
and  the  WSIC  PDU  logger. 

-  The  latency  of  the  target  entity  state  data  from  the  SIMLAB  PDU  logger  into  the 
SIMLAB  simulation  were  determined  by  matching  the  north  velocity  components  of 
the  target  in  the  PDU  data  with  the  corresponding  data  logged  in  the  SIMLAB 
simulation.  The  results  are  given  in  Table  4.3.3.1-13.  Comparing  Table  4.3.3.1-13 
with  Table  4.3.3.1-10  shows  that  the  latency  of  the  target  data  between  the  SIMLAB 
PDU  logger  and  the  SIMLAB  simulation  was  more  than  four  times  that  between  the 
WSIC  simulation  and  the  WSIC  PDU  logger.  The  total  latency  between  the  WSIC 
simulation  and  the  SIMLAB  simulation  shows  that  the  SIMLAB  simulation  was 
usually  running  about  one  PDU  update  time  (50  msec)  behind  the  WSIC  simulation. 
However,  there  were  instances  during  a  run  in  which  an  individual  frame  of  the 
SIMLAB  simulation  was  as  many  as  seven  PDU  update  times  (350  msec)  behind  the 
WSIC  simulation. 

—  Comparing  Table  4.3.3.1-13  with  Table  4.3.3.1-12  shows  that  the  total  latency  of 
die  target  data  between  the  WSIC  simulation  and  the  SIMLAB  simulation  was 
about  the  same  as  that  between  the  WSIC  simulation  and  the  WSSF  simulation, 
about  70  msec.  As  noted  previously,  most  of  this  latency  was  between  the  NIU  at 
the  receiving  node  and  the  simulation  at  that  node. 

—  When  the  target’s  north  velocity  component  was  matched,  it  was  noted  that  the 
target  velocity  was  not  being  updated  in  the  SIMLAB  simulation  at  the  same  rate 
as  the  target  positional  data.  This  resulted  when  an  updated  target  PDU  was  not 
received  by  the  SIMLAB  NIU  by  the  time  data  was  to  be  logged  by  the  SIMLAB 
simulation,  just  as  noted  above  for  the  target  data  received  by  the  WSSF.  As  a 
result,  the  target  velocity  data  were  occasionally  updated  at  a  low  rate  (as  low  as 
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~3  Hz).  This  was  an  example  of  an  ADS-induced  error  and  was  evaluated  under 
Test  Objective  4-2  (Section  4.4.2.3). 

-  Missile  Entity  State  Data  Latencies 

-  The  mean  latencies  of  the  missile  entity  state  data  were  small  and  consistent  run-to-run 
(Table  4.3.3.1-11),  with  the  exception  of  Run  #12.  The  mean  latencies  between  the 
SIMLAB  simulation  and  the  SIMLAB  PDU  logger  were  about  the  same  as  those  for 
the  lowest  latency  run  from  the  V&V  Mission.  The  mean  latencies  between  the 
SIMLAB  PDU  logger  and  the  WSIC  PDU  logger  were  essentially  the  same  as  for  the 
V&V  Mission. 

-  As  Table  4.3.3.1-11  shows,  if  Run  #12  is  excluded,  the  mean  latency  between  the 
SIMLAB  simulation  and  the  SIMLAB  PDU  logger  was  less  than  half  of  the  mean 
transmission  latency  between  PDU  loggers  and  was  about  the  same  as  the  mean 
latency  of  the  target  entity  state  data  between  the  WSIC  simulation  and  the  WSIC 
PDU  logger. 

~  As  Figure  4.3.3.1-23  shows,  the  latencies  primarily  fell  into  a  single  group.  However, 
there  were  a  number  of  higher  latency  “spikes”  falling  outside  the  group. 

-  Table  4.3.3.1-11  shows  that  Run  #12  had  an  unusually  high  mean  latency.  Figure 
4.3.3.1-24  shows  that  the  latency  on  this  run  had  a  higher  baseline  and  a  larger  number 
of  high  latency  “spikes”  compared  to  Run  #9  (Fig.  4.3.3.1-23).  Note  that  the  “spikes” 
on  Run  #12  were  distributed  throughout  the  missile  flyout. 

—  The  mean  latency  of  the  missile  entity  state  data  between  the  WSIC  PDU  logger  and 
the  WSIC  simulation  was  checked  and  found  to  be  about  50%  larger  than  the  mean 
latency  of  the  target  entity  state  data  between  the  SIMLAB  PDU  logger  and  the 
SIMLAB  simulation  (63  msec  vs.  45  msec).  The  missile  data  were  not  used  by  the 
WSIC  simulation;  they  were  only  logged  there.  These  latency  values  do  not 
necessarily  represent  the  latency  of  inputting  the  missile  data  into  the  WSIC 
simulation. 

SUMMARY.  The  mean  latencies  of  all  entity  state  data  were  relatively  small  and  consistent  run- 
to-run.  The  mean  transmission  latencies  between  the  simulation  nodes  were  essentially  the  same 
as  for  the  V&V  Mission,  about  20  msec  for  all  the  node  pairs.  The  mean  latency  of  the  target 
and  missile  entity  state  data  between  their  creating  simulations  and  the  first  PDU  logger  was 
generally  less  than  half  of  the  mean  transmission  latency  between  nodes.  However,  the  mean 
latency  of  the  shooter  entity  state  data  between  the  WSSF  simulation  and  the  WSSF  PDU  logger 
was  more  than  twice  the  mean  transmission  latency.  The  mean  latencies  of  the  target  and  missile 
entity  state  data  between  the  PDU  logger  and  simulation  at  the  receiving  node  were  more  than 
twice  the  mean  latencies  between  the  simulation  and  the  PDU  logger  at  the  creating  node.  The 
mean  latencies  between  any  pair  of  nodes  still  exhibited  relatively  large  sample-to-sample 
variations,  and  high  latency  “spikes”  were  still  observed,  even  between  the  PDU  loggers. 
However,  some  of  the  “spikes”  may  have  been  caused  by  delays  in  time  stamping  the  PDU  data  at 
the  loggers. 
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4.3.3.2  Latency  Effects 


4.3.3.2.1  Entity  Presentation  Errors 

As  noted  in  Sections  4.1. 1.2  and  4.1. 1.3,  the  random  variations  in  latency  between  the  WSIC  and 
the  SIMLAB  during  a  run  resulted  in  an  uncertainty  in  the  target  location,  as  perceived  at  the 
SIMLAB.  This  complicated  the  dynamic  verification  of  the  target  presentation  in  the  SIMLAB 
and  prevents  the  implementation  of  a  deterministic  real-time  correction  for  latency  effects.  Table 
4. 1.1. 3-3  gives  the  results  of  applying  Equation  1  to  the  latency  characteristics  of  the  Parametric 
Study  Mission  runs  and  shows  that  the  average  of  the  mean  uncertainties  of  the  target  position  for 
the  six  complete  V2  runs  was  about  32  ft.  This  is  significantly  larger  than  the  lethal  radius  of  the 
missile,  so  that  lethality  results  from  the  LSP  configuration  cannot  be  considered  valid.  Also,  note 
that  the  uncertainty  value  quoted  is  based  on  the  standard  deviation  of  the  latency.  Some  of  the 
high  latency  “spikes”  were  up  to  ten  times  the  standard  deviation,  so  that  the  observed  target 
position  might  instantaneously  diverge  from  its  correct  location  by  over  300  fit. 

Some  of  the  large  latency  values  caused  uncharacteristically  large  time  intervals  between  target 
entity  state  updates  at  the  receiving  nodes.  The  normal  time  between  creating  target  PDUs  at  the 
WSIC  was  about  50  msec,  but  occasionally  large  latencies  would  prevent  the  target  PDUs  from 
being  updated  at  the  receiving  nodes  for  several  hundred  milliseconds.  When  such  large  time  gaps 
between  updates  occurred,  the  target  velocity  data  input  into  either  the  WSSF  or  the  SIMLAB 
simulation  would  not  be  updated  during  the  gap  (however,  the  target  position  was  “updated” 
during  the  gap  by  dead  reckoning,  so  that  the  target  position  and  velocity  were  inconsistent  with 
each  other  during  the  gap).  When  the  next  updated  target  PDU  was  received,  a  discontinuous 
adjustment  of  the  target  velocity  would  occur. 

-  The  impact  of  this  on  the  WSSF  was  that  the  closing  velocity  between  the  shooter  and 
target,  vc,  was  observed  by  the  shooter  pilot  to  “jump”  at  times. 

-  The  impact  of  this  on  the  SIMLAB  simulation  was  aggravation  of  the  latitude  divergence 
problem  (see  Section  4. 1.1. 3),  since  the  SIMLAB  simulation  integrated  the  target  velocity 
to  determine  the  target  position. 

Large  gaps  in  updating  the  PDU  data  at  the  receiving  nodes  were  also  caused  by  irregularities  in 
the  PDU  creation  rate  (see  Section  4.4.2. 3). 

Random  latency  variations  were  also  evident  in  the  data  for  the  other  entities.  Hence,  the  shooter 
and  the  missile  positions  perceived  at  other  nodes  were  likewise  uncertain,  and  their  velocities 
exhibited  discontinuities.  Since  the  missile  had  a  much  higher  velocity  than  the  target  (~3  times 
larger),  the  uncertainty  in  its  position,  as  perceived  by  the  target,  was  proportionately  larger. 

Besides  discontinuities  in  entity  presentation,  the  high  latency  “spikes”  can  affect  the  overall 
simulation  performance,  if  they  occur  during  critical  functions.  This  may  have  caused  the 
SIMLAB  Carco  table  to  exceed  its  gimbal  limits  on  some  of  the  runs  (because  the  target 
presentation  exceeded  the  allowable  seeker  field-of-view).  When  this  happened,  the  missile 
simulation  was  immediately  terminated  before  the  missile  could  complete  its  simulated  flyout. 
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4.3.3.2.2  Launch  Conditions  Differences  Between  Nodes 

The  launch  conditions  were  determined  from  data  collected  at  the  various  logging  locations  and 
using  the  procedures  discussed  in  Section  4.3.2.  The  Time  of  Flare  Release  and  the  Time  of 
Evasive  Maneuver  were  not  determined,  because  these  applied  to  scenarios  from  the  planned 
Latency  Study  Mission  which  were  not  executed.  Results  are  expressed  as  differences  in  the 
launch  condition  parameters  as  determined  for  the  various  nodes. 

V&V  Mission 


STMT.AB  Simulation  Data  Compared  to  SIMLAB  Logged  PDU  Data  .  The  differences  between 
the  launch  conditions  from  the  PDUs  logged  at  the  SIMLAB  and  those  from  the  SIMLAB 
simulation  were  computed  and  are  given  in  Table  4.3.3.2.2-1.  The  launch  conditions  as  logged 
from  the  SIMLAB  simulation  were  generally  accepted  as  the  set  of  launch  conditions  for  each  run 
and  were  the  source  of  data  for  Test  Objective  2.  Note  that  three  relative  latencies  are  given 
(these  are  differences  in  latencies  of  data  logged  at  PDU  logger  and  data  logged  in  simulation): 
the  latency  of  the  Fire  (Missile)  PDUs  logged  at  the  SIMLAB  PDU  logger  (reflecting  latency 
between  the  SIMLAB  simulation  and  the  SIMLAB  PDU  logger),  the  relative  latency  of  the 
shooter  entity  state  data,  and  the  relative  latency  of  the  target  entity  state  data.  Differences  are 
not  given  for  shooter  velocity,  target  velocity,  and  target  altitude;  these  differences  were  always 
minimal,  because  of  the  nature  of  the  LPN-15  scenario. 


Table  4.3.3.2.2-1.  Differences  Between  Launch  Conditions  from  PDUs  Logged  at  SIMLAB 
and  Launch  Conditions  from  SIMLAB  Simulation  (10/29/96) 


Relative  Latencies  (ms) 

Run  # 

Range  (ft) 

Sh  Alt  (ft) 

Aspect 

(°) 

Lead  (°) 

FIRE 

Shooter 

Target 

3 

-86.6 

-9.8 

0.87 

0.04 

90 

-35 

-55 

4 

-89.3 

-9.3 

0.65 

-0.29  ! 

109 

-31 

-22 

5 

-170.7 

-18.2 

1.52 

0.32 

253 

-18 

-27 

7 

-208.8 

-16.7 

2.08 

-1.39 

412 

-8 

-31 

8 

-274.5 

-21.5 

2.29 

-1.25 

463 

51 

-25 

10 

-68.0 

-1.9 

0.22 

0.07 

35 

-23 

0 

15 

-61.1 

-2.6 

0.22 

0.07 

33 

23 

-7 

16 

-38.2 

-2.4 

0.52 

-0.22 

33 

14 

-33 

18 

-255.9 

-65.1 

3.23 

-1.92 

676 

20 

-52 

19 

154.5 

-2.5 

2.40 

-1.50 

441 

473 

15 

20 

-241.3 

-46.1 

2.37 

-1.45 

471 

-44 

-28 

23 

-330.3 

-74.1 

4.61 

-1.39 

822 

7 

-37 

25 

-246.0 

-66.5 

4.71 

1.17 

570 

109 

-78 

26 

-258.5 

-47.9 

3.27 

!  -1.88 

579 

44 

-8 

The  following  is  noted  from  Table  4.3. 3. 2.2-1: 
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-  The  differences  between  the  launch  conditions  determined  from  the  PDUs  and  tho  se  from 
the  SIMLAB  simulation  generally  increased  as  the  latency  of  the  Fire  (Missile)  PDU 
increased.  This  relationship  was  generally  true  because  the  latency  of  the  Fire  (Missile) 
PDU  usually  dominated  the  relative  latencies  of  the  shooter  and  target  entity  state  data. 

-  In  general,  the  launch  range  from  the  PDU  data  was  smaller  than  that  from  the  simulation 
data.  This  occurred  because  the  shooter  was  closing  on  the  target,  and  the  PDU  data  used 
to  determine  the  launch  conditions  corresponded  to  a  later  time  than  for  the  simulation 
data  (the  launch  conditions  were  determined  from  the  logged  PDU  data  when  the  Fire 
(Missile)  PDU  which  was  generated  in  the  simulation  was  received  by  the  PDU  logger). 

-  The  shooter  altitude  from  the  PDU  data  was  lower  than  that  from  the  simulation  data 
because  the  shooter  was  losing  altitude  with  time,  and  the  PDU  data  corresponded  to  a 
later  time. 

-  The  target  aspect  angle  from  the  PDU  data  was  larger  than  that  from  the  simulation  data 
because  the  turning  of  the  target  caused  the  aspect  angle  to  increase  with  time,  and  the 
PDU  data  corresponded  to  a  later  time. 

-  The  lead  angle  depended  on  how  the  shooter  was  maneuvering  prior  to  the  launch.  This 
varied  slightly  from  run-to-run,  so  that  a  clear  trend  in  the  lead  angle  was  not  obvious. 

-  The  exception  to  the  trend  of  the  launch  range  differences  increasing  with  Fire  (Missile) 
PDU  latency  was  Run  #19.  On  this  run  the  shooter  entity  state  data  had  a  much  larger 
latency  at  die  PDU  logger  than  into  the  SIMLAB  simulation  (the  simulation  and  the 
logger  used  the  same  set  of  shooter  entity  state  data  because  update  time  of  the  logged 
shooter  entity  state  PDUs  was  abnormally  long),  while  the  target  entity  state  data  had 
about  the  same  latency.  In  this  case,  the  shooter  data  from  PDUs  represented  a  much 
earlier  position  relative  to  the  target,  so  that  the  launch  range  from  the  PDU  data  was 
longer  than  that  from  the  SIMLAB  simulation. 

-  The  launch  range  differences  in  Table  4.3.3.2.2-1  are  plotted  versus  the  Fire  (Missile) 
PDU  latency  in  Figure  4.3.3.2.2-1,  excluding  the  data  for  Run  #19.  This  figure  illustrates 
the  general  trend  noted  above.  Note  that  the  linear  fit  to  the  data  in  the  figure  does  not 
pass  through  the  origin  (i.e.,  zero  Fire  (Missile)  PDU  latency  did  not  correspond  to  a  zero 
launch  range  difference).  This  was  because  the  launch  range  differences  were  also  due  to 
the  shooter  and  target  entity  state  data  latencies  (i.e.,  if  the  Fire  (Missile)  PDU  latency  was 
reduced  to  zero  there  would  still  be  latencies  in  the  shooter  and  target  data).  Figure 
4.3.3.2.2-1  shows  that  Fire  (Missile)  PDU  latencies  need  to  be  less  than  100  msec  if  the 
differences  in  launch  range  are  to  be  less  than  100  ft. 

SIMLAB  Simulation  Data  Compared  to  WSSF  Simulation  Data  .  The  differences  between  the 
launch  conditions  from  the  SIMLAB  simulation  and  those  from  the  WSSF  simulation  were 
computed  and  are  given  in  Table  43.3.2.2-2.  The  launch  conditions  from  the  SIMLAB 
simulation  were  logged  for  each  run.  However,  the  launch  conditions  for  the  WSSF  were  not 
computed  by  the  simulation  and  had  to  be  determined  after  the  test  as  follows: 

-  The  launch  event  at  the  WSSF  was  determined  by  using  the  shooter  altitude  from  the  F/A- 
18  cockpit  data  display  which  appeared  at  trigger  squeeze. 

-  The  shooter  simulation  data  frame  logged  at  the  WSSF  which  had  the  same  shooter 
altitude  was  located,  and  other  shooter  parameters  were  noted. 
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—  The  target  simulation  data  frame  logged  at  the  WSSF  with  the  same  IRIG  time  as  the 
shooter  frame  was  located,  and  the  target  parameters  were  noted. 

-  The  launch  conditions  were  computed  from  the  shooter  and  target  data. 

For  this  comparison,  the  differences  were  found  to  relate  primarily  to  the  latency  of  the  shooter 
entity  state  data  received  at  the  SIMLAB  relative  to  the  latency  of  the  launch  indication  received 
at  the  SIMLAB  from  the  WSSF.  The  entries  for  relative  SIMLAB  latencies  in  Table  4.33.2.2-2 
are  the  differences  between  the  latency  of  the  shooter  entity  state  data  received  at  the  SIMLAB 
and  the  latency  of  the  SIMLAB  launch  indication  (i.e.,  differences  of  the  two  previous  columns) 


Table  43.3.2.2-2.  Differences  Between  Launch  Conditions  from  SIMLAB  Simulation  and 
Launch  Conditions  from  WSSF  Simulation  (10/29/96) 


SIMLAB  Latencies  (ms) 

Run  # 

Range  (ft) 

Sh  Alt  (ft) 

Aspect 

(°) 

Lead  (°) 

Launch 

Shooter 

Relative 

3 

155.2 

5.5 

1.32 

-1.09 

-33 

67 

100 

4 

160.8 

4.5 

2.12 

-0.94 

-18 

83 

101 

5 

163.5 

4.6 

1.91 

-1.64 

-20 

80 

100 

7 

161.1 

2.0 

1.01 

-1.08 

2 

55 

53 

8 

305.5 

15.8 

0.1 

-0.52 

-25 

328 

353 

10 

343.8 

9.2 

0.1 

-0.71 

6 

453 

447 

15 

557.3 

46.1 

0.82 

-0.46 

-21 

679 

700 

16 

491.1 

42.8 

0.24 

-0.17 

-30 

621 

651 

18 

550.9 

69.9 

0.32 

-0.56 

-7 

745 

752 

19 

519.0 

47.9 

0.51 

-0.53 

8 

602 

594 

20 

562.8 

61.5 

0.11 

-0.47 

11 

759 

748 

23 

582.4 

63.9 

-0.31 

-1.19 

-9 

743 

752 

25 

679.6 

82.1 

-1.17 

-3.50 

-356 

693 

1049 

26 

544.6 

53.9 

0.53 

-0.41 

15 

716 

701 

The  following  is  noted  from  Table  4.33.2.2-2: 

-  The  latencies  of  the  launch  indication  received  at  the  SIMLAB  were  much  smaller  than  the 
latencies  of  the  shooter  entity  state  data.  Also,  the  latencies  of  the  launch  indication  were 
about  the  same  run-to-run,  with  the  exception  of  Run  #25.  This  occurred  because  the 
launch  indication  was  sent  from  the  WSSF  to  the  SIMLAB  over  a  dedicated  link  as  part 
of  the  pre-launch  SMS  data,  so  that  it  did  not  share  the  link  with  the  entity  state  data.  In 
other  words,  the  SMS  data  was  significantly  out  of  synchronization  with  the  shooter  entity 
state  data.  Since  the  launch  signal  was  generated  at  the  WSSF,  the  negative  latencies  to 
the  SIMLAB  do  not  make  physical  sense  and  probably  indicate  a  timing  problem. 

-  Because  the  launch  indication  latency  was  much  smaller  than  the  shooter  entity  state  data, 
die  SIMLAB  utilized  relatively  older  shooter  entity  state  data  at  time  of  launch,  when  the 
shooter  was  at  a  larger  range  from  the  target  and  at  a  higher  altitude.  This  is  reflected  in 


4-137 


the  positive  differences  in  the  launch  ranges  and  shooter  altitudes  determined  by  the 
SIMLAB  as  compared  to  these  parameters  determined  by  the  WSSF. 

-  The  target  aspect  angle  determined  by  the  SIMLAB  w  as  generally  larger  than  that 
determined  by  the  WSSF  because  the  latency  of  the  target  data  into  the  SIMLAB 
simulation  was  usually  significantly  less  than  that  into  the  WSSF  simulation.  As  a  result, 
the  SIMLAB  perceived  a  more  recent  target  bearing.  Since  the  turning  of  the  target 
caused  the  aspect  angle  to  increase  with  time,  the  SIMLAB  determined  a  larger  angle. 

-  The  launch  range  differences  in  Table  4.33.2.2-2  are  plotted  versus  the  latency  of  the 
SIMLAB  launch  indication  relative  to  the  latency  of  the  shooter  entity  state  data  in  Figure 

4.33.2.2- 2.  This  figure  illustrates  the  trend  noted  above.  Note  that  the  linear  fit  to  the 
data  in  the  figure  again  does  not  pass  through  the  origin  (as  for  Fig.  4.33.2.2-1).  Figure 

433.2.2- 2  shows  that  other  latencies  (primarily  the  latency  of  the  target  entity  state  data 
to  the  WSSF  simulation)  prevented  achieving  launch  range  differences  of  less  than  100  ft. 

WSIC  Logged  PDU  Data  Compared  to  WSSF  Simulation  Data  .  The  differences  between  the 
launch  conditions  from  the  PDUs  logged  at  the  WSIC  and  those  from  the  WSSF  simulation  were 
computed  and  are  given  in  Table  433.2.2-3.  This  comparison  helped  to  quantify  the  effects  of 
latency  between  the  shooter  and  the  target  simulations.  Unfortunately,  shooter  entity  state  data 
were  not  logged  in  the  WSIC  simulation  during  the  V&V  Mission,  so  that  a  direct  comparison 
between  the  WSIC  and  WSSF  simulations  was  not  possible.  As  in  Table  433.2.2-2,  the 
differences  were  found  to  relate  primarily  to  the  latency  of  the  shooter  entity  state  data  logged  at 
the  WSIC  relative  to  the  latency  of  the  Fire  (Missile)  PDUs  logged  at  the  WSIC.  The  entries  for 
relative  WSIC  latencies  in  Table  433.2.2-3  are  the  differences  between  the  latency  of  the  shooter 
entity  state  data  logged  on  the  WSIC  PDU  logger  and  the  latency  of  the  Fire  (Missile)  PDUs 
logged  on  the  WSIC  PDU  logger  (i.e.,  differences  of  the  two  previous  columns). 


Table  43.3.2.2-3.  Differences  Between  Launch  Conditions  from  PDUs  Logged  at  WSIC 
and  Launch  Conditions  from  WSSF  Simulation  (10/29/96) 


WSIC  Latencies  (ms) 

Run  # 

Range  (ft) 

Sh  Alt  (ft) 

Aspect 

(°) 

Lead  (°) 

FIRE 

Shooter 

Relative 

3 

67.1 

-5.1 

2.27 

-1.09 

107 

67 

-40 

4 

82.7 

-5.7 

3.02 

-1.39 

125 

83 

-42 

5 

-0.5 

-14.1 

3.65 

-1.48 

272 

80 

-192 

7 

-45.1 

-15.0 

3.21 

-2.59 

435 

59 

-376 

8 

29.3 

-6.2 

2.54 

-1.88 

482 

386 

-96 

10 

281 

7.2 

0.44 

-0.74 

51 

453 

402 

15 

498 

43 

1.13 

-0.43 

49 

661 

16 

456.3 

39.9 

0.91 

-0.49 

51 

641 

590 

18 

303.6 

4.7 

3.67 

-2.57 

689 

776 

87 

19 

696.2 

45.3 

3.19 

-2.11 

464 

1151 

687 

20 

328.5 

15.4 

2.58 

-1.99 

560 

804 

244 

23 

236.9 

-21.6 

5.33 

-2.97 

937 

743 

-194 

25 

441.7 

14.9 

3.70 

-2.44 

587 

809 

222 
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|  26 

294.8 

6.0 

3.93 

-2.38 

596 

754 

158 

The  following  is  noted  from  Table  4.3. 3. 2.2-3: 

-  The  latencies  of  the  Fire  (Missile)  PDUs  received  at  the  WSIC  were  sometimes  smaller 
and  sometimes  larger  than  the  latencies  of  the  shooter  entity  state  data. 

-  The  signs  of  the  shooter  altitude  difference  agreed  with  the  signs  of  the  relative 
latencies,  as  would  be  expected  (positive  relative  latency  corresponds  to  older  shooter 
entity  state  data  when  the  shooter  was  at  a  higher  altitude  and  vice  versa). 

-  In  those  cases  in  which  the  relative  latency  was  positive,  positive  launch  range 
differences  resulted,  as  in  Table  43.3.2.2-2.  However,  negative  relative  latencies 
resulted  in  either  positive  or  negative  launch  range  differences;  other  latencies 
appeared  to  be  a  factor  here. 

—  The  differences  in  target  aspect  angle  appeared  to  be  related  to  the  Fire  (Missile)  PDU 
latency  itself,  as  in  Table  4.3.3.2.2-1.  Note  that  Fire  (Missile)  PDU  latencies  must  be 
less  than  about  50  msec  if  the  differences  in  aspect  angle  are  to  be  less  than  f. 

-  The  launch  range  differences  in  Table  4.33.2.2-3  are  plotted  versus  the  latency  of  the 
shooter  entity  state  data  relative  to  the  latency  of  the  WSIC  Fire  (Missile)  PDU  in  Figure 

433.2.2- 3.  This  figure  also  contains  the  launch  range  differences  plotted  in  Figure 

433.2.2- 2  and  illustrates  the  general  trend  noted  above.  Note  that  the  linear  fit  to  the 
data  in  the  figure  again  does  not  pass  through  the  origin  (as  for  Figs.  433.2.2-1  and 

433.2.2- 2)  and  parallels  the  linear  fit  from  Figure  433.2.2-2.  Figure  433.2.2-3  shows 
that  negative  relative  latencies  (Fire  (Missile)  PDU  latency  greater  than  shooter  PDU 
latency)  with  magnitudes  greater  than  100  msec  are  needed  if  the  differences  in  launch 
range  are  to  be  less  than  100  ft.  In  other  words,  the  Fire  (Missile)  PDU  latency  would 
have  to  be  greater  than  100  msec  to  achieve  launch  range  differences  less  than  100  ft. 
This  is  not  consistent  with  the  conclusion  above  that  Fire  (Missile)  PDU  latencies  must  be 
less  than  50  msec  for  the  differences  in  the  target  aspect  angle  to  be  less  than  1  °.  This 
inconsistency  is  due  to  the  target  entity  state  data  latencies  not  being  considered.  The 
implication  here  is  that  the  latencies  for  all  entity  state  data  and  for  the  Fire  (Missile)  PDU 
must  be  less  than  about  50-100  msec  to  achieve  good  agreement  between  the  shooter  and 
target  on  all  launch  conditions  (launch  range  within  100  ft,  shooter  altitude  within  10  ft, 
and  target  aspect  angle  and  lead  angle  within  F). 

WSIC  Logged  PDU  Data  Compared  to  WSSF  Logged  PDU  Data  .  The  differences  between  the 
launch  conditions  from  the  WSIC  PDU  data  and  those  from  the  WSSF  PDU  data  were  computed 
and  are  given  in  Table  433.2.2-4.  This  table  shows  how  well  the  PDU  data  logged  at  various 
locations  agreed,  because  this  particular  comparison  resulted  in  the  largest  differences  for  any  pair 
of  PDU  loggers.  Relative  latencies  are  given  in  the  table,  as  in  Table  433.2.2-1. 

Table  433.2.2-4  shows  excellent  agreement  for  all  launch  conditions.  This  was  due  to  the  small 
latencies  (mean  values  of  ~20  msec)  between  the  PDU  loggers  during  the  V&V  Mission.  The 
small  logger-to-logger  latencies  resulted  in  small  relative  latencies  in  Table  43.3.2.24.  This  table 
reinforces  the  conclusion  that  latencies  less  than  about  50-100  msec  were  required  for  good 
agreement  between  the  shooter  and  target  on  all  launch  conditions. 
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Table  4.3.3.2.2-4.  Differences  Between  Launch  Conditions  from  PDUs  Logged  at  WSIC 
and  Launch  Conditions  from  PDUs  Logged  at  WSSF  (10/29/96) 


Relative  Latencies 

(ms) 

Run  # 

Range  (ft) 

Sh  Alt  (ft) 

Aspect 

O 

Lead  (°) 

FIRE 

Shooter 

Target 

3 

8.8 

0.3 

0.06 

-0.07 

6 

11 

-13 

4 

22.4 

0.6 

0.21 

-0.15  ! 

6 

15 

-28 

5 

18.4 

0.7 

0.19 

-0.13 

10 

19 

-21 

7  n 

15.0 

0.2 

0.11 

-0.09 

16 

23 

-18 

8 

11.7 

0.4 

0.12 

-0.10 

7 

15 

-13 

10 

18.0 

0.2 

0.09 

-0.10 

5 

19 

-13 

15 

76.2 

7.0 

0.01 

-0.09 

-22 

78  1 

-24 

16 

15.5 

0.7 

0.10 

-0.10 

.  6 

16 

-9 

18 

12.8 

0.5 

0.10 

-0.07 

6 

12 

-11 

19 

26.6 

0.7 

0.24 

-0.16 

15 

24 

-23 

20 

5.3 

0.03 

0.07 

-0.05 

79 

79 

56 

23 

13.8 

1.0 

0.06 

-0.05 

17 

17 

-12 

25 

17.4 

0.9 

0.10 

-0.07 

17 

17 

-16 

26 

6.9 

0 

0.10 

-0.07 

6 

6 

-11 

Mean 

19.2 

0.9 

0.11 

-0.09 

10.9 

25.1 

-11.1 

StdDev 

17.4 

1.8 

0.06 

0.03 

21.5 

23.1 

23.1 

CONCLUSION .  Large  latencies  (>100  msec)  result  in  significant  differences  between  the  launch 
conditions  determined  at  the  various  nodes.  Significant  differences  were  observed  between  the 
launch  conditions  determined  by  the  SIMLAB  simulation  as  compared  to  the  WSSF  simulation. 
These  differences  were  due  to  the  relatively  large  latencies  between  the  simulations  and  the  PDU 
logger  at  the  same  node.  When  launch  conditions  determined  from  PDU  data  were  compared  for 
the  various  nodes,  there  was  excellent  agreement,  due  to  the  small  latencies  between  PDU 
loggers.  Total  latencies  (from  simulation  to  simulation)  for  all  entity  state  data  and  for  the  Fire 
(Missile)  PDU  must  be  less  than  about  50-100  msec  to  achieve  good  agreement  between  the 
shooter  and  target  on  all  launch  conditions  (good  agreement  is  defined  to  represent  achievable 
differences:  launch  range  within  100  ft,  shooter  altitude  within  10  ft,  and  target  aspect  angle  and 
lead  angle  within  1  °;  most  of  these  differences  are  10%,  or  less,  of  the  shot  box  tolerances  given 
in  Table  4. 1.1. 2-1). 
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Figure  4.3.3.2.2-2.  Launch  Range  from  SIMLAB  Simulation  Data  Relative  to  Launch  Range  from  WSSF  Simulation  Data  vs. 
Shooter  Entity  State  Data  Latency  at  SIMLAB  Relative  to  SIMLAB  Launch  Indication  Latency  (10/29/96) 


Parametric  Study  Mission 


SIMLAB  Simulation  Data  Compared  to  WSSF  Simulation  Data  .  The  differences  between  the 
launch  conditions  from  the  SIMLAB  simulation  and  those  from  the  WSSF  simulation  were 
computed  and  are  given  in  Table  4.3.3.2.2-5.  The  launch  conditions  from  the  SIMLAB 
simulation  were  logged  for  each  run.  However,  the  launch  conditions  for  the  WSSF  were  not 
computed  by  the  simulation  and  had  to  be  determined  after  the  test  as  for  the  V&V  Mission.  The 
entries  for  relative  latencies  in  Table  4.33.2.2-5  are  the  differences  between  the  latency  of  each 
type  of  data  (i.e.,  launch  indication,  shooter  entity  state,  or  target  entity  state  data)  received  at  the 
SIMLAB  simulation  and  the  latency  of  the  same  type  received  at  the  WSSF  simulation. 
However,  note  that  in  this  case  the  relative  launch  indication  latency  and  the  relative  shooter 
entity  state  data  latency  are  only  between  the  WSSF  and  the  SIMLAB. 


Table  43.3.2.2-5.  Differences  Between  Launch  Conditions  from  SIMLAB  Simulation  and 
Launch  Conditions  from  WSSF  Simulation  (11/19/96) 


Relative  Latencie: 

(ms) 

Run  # 

Range  (ft) 

Sh  Alt  (ft) 

Aspect 

(°) 

Lead  (°) 

Launch 

Shooter 

Target 

9 

34.7 

2.9 

0 

-0.27 

69 

78 

30 

10 

26.3 

0.5 

-0.16 

bbh 

31 

79 

179 

12 

97.6 

2.7 

-0.15 

1 

31 

59 

8 

18 

45.4 

4.7 

0.49 

0.02 

17 

118 

7 

19 

29.4 

4.7 

-0.69 

0.04 

17 

117 

107 

22 

34.6 

1.7 

-0.45 

-0.43 

9 

108 

71 

Mean 

44.7 

2.9 

-0.16 

-0.18 

29.0 

93.2 

67 

Std  Dev 

26.7 

1.7 

0.40 

0.18 

21.4 

24.5 

67.2 

Table  4.33.2.2-5  shows  excellent  agreement  for  all  launch  conditions.  This  was  due  to  the  small 
latencies  between  the  SIMLAB  simulation  and  the  WSSF  simulation  during  the  Parametric  Study 
Mission.  This  table  shows  that  latencies  less  than  100  msec  resulted  in  excellent  agreement 
between  the  shooter  and  missile  on  all  launch  conditions. 

WSIC  Simulation  Data  Compared  to  WSSF  Simulation  Data  .  The  differences  between  the  launch 
conditions  from  the  WSIC  simulation  and  those  from  the  WSSF  simulation  were  computed  and 
are  given  in  Table  433.2.2-6.  The  launch  conditions  for  the  WSSF  were  not  computed  by  the 
simulation  and  had  to  be  determined  after  the  test  as  for  the  V&V  Mission.  Also,  the  launch 
conditions  for  the  WSIC  were  not  computed  by  the  simulation  and  had  to  be  determined  after  the 
test  as  follows: 

-  The  launch  event  at  the  WSSF  was  determined  by  using  the  shooter  altitude  from  the  F/A- 
18  cockpit  data  display  which  appeared  at  trigger  squeeze. 

-  The  shooter  data  frame  logged  by  the  WSIC  simulation  which  had  a  shooter  altitude 
closest  to  this  value  was  located,  and  other  shooter  parameters  were  noted,  along  with 
the  IRIG  time  of  this  frame. 
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—  The  target  data  frame  logged  by  the  WSIC  simulation  with  the  same  IRIG  time  as  the 
shooter  frame  was  located,  and  the  target  parameters  were  noted. 

The  launch  conditions  were  computed  from  the  shooter  and  target  data. 

The  entries  for  relative  latencies  in  Table  4.33.2.2-6  are  the  differences  between  the  latency  of 
each  type  of  data  received  at  the  WSIC  simulation  and  the  latency  of  the  same  type  received  at  the 
WSSF  simulation.  However,  note  that  in  this  case  the  relative  launch  indication  latency  is  only  for 
data  transmitted  from  the  WSSF  to  the  WSIC,  the  relative  shooter  entity  state  data  latency  is  only 
for  data  transmitted  from  the  WSSF  to  the  WSIC,  and  the  relative  target  entity  state  data  latency 
is  only  for  data  transmitted  from  the  WSIC  to  the  WSSF  (these  latencies  are  negative  in  Table 
43.3.2.2-6  because  the  target  data  flow  was  from  the  WSIC  to  the  WSSF,  the  opposite  of  the 
other  data  flows). 


Table  43.3.2.2-6.  Differences  Between  Launch  Conditions  from  WSIC  Simulation  and 
Launch  Conditions  from  WSSF  Simulation  (11/19/96) 


Relative  Latencies  (ms)  j 

Run# 

Range (ft) 

Sh  Alt  (ft) 

■1 

Lead  (°) 

Launch 

Shooter 

Target 

9 

42.0 

-4.4 

1.16 

-0.52 

236 

146 

-21 

10 

51.7 

-3.3 

2.34 

-0.59 

99 

147 

-14 

12 

126.2 

-3.7 

1.01 

-0.80 

194 

122 

-72 

18 

99.4 

-1.2 

1.20 

-0.51 

174 

174 

-22 

19 

65.7 

-5.2 

0.67 

-0.50 

174 

122 

-12 

22 

105.5 

-4.2 

1.32 

-1.00 

238 

185 

-9 

Mean 

81.8 

-3.7 

1.28 

-0.65 

185.8 

149.3 

-25 

Std  Dev 

33.4 

1.4 

0.56 

0.20 

51.2 

26.0 

23.6 

Table  4.33.2.2-6  shows  good  agreement  for  all  launch  conditions.  Note  that  relative  latencies  in 
this  table  are  larger  than  those  in  Table  433.2.2-5  and  that  the  differences  in  launch  conditions  in 
Table  4.33.2.2-6  are  correspondingly  larger.  Also  note  that  the  target  entity  state  data  latency 
between  the  WSIC  and  the  WSSF  at  launch  was  smaller  than  the  mean  value  for  data  passed 
before  launch  (Table  4.3.3.1-12),  but  the  latencies  of  the  launch  indication  and  the  shooter  entity 
state  data  passed  in  the  opposite  direction  were  much  larger.  Nevertheless,  even  with  relative 
latencies  up  to  about  200  msec,  the  agreement  in  the  launch  conditions  was  still  good.  (Note  that 
these  latency  values  represent  a  relaxation  of  the  latency  requirements  implied  by  the  V&V 
Mission  results,  50-100  msec.  Apparently,  the  combination  of  low  latencies  for  all  data  sources 
resulted  in  less  stringent  latency  requirements.) 

wstr  Simulation  Data  Compared  to  SIMLAB  Simulation  Data  .  The  differences  between  the 
launch  conditions  from  the  WSIC  simulation  and  those  from  the  SIMLAB  simulation  were 
computed  and  are  given  in  Table  4.33.2.2-7.  The  launch  conditions  from  the  SIMLAB  simulation 
were  logged  for  each  run.  However,  the  launch  conditions  for  the  WSIC  were  not  computed  by 
the  simulation  and  had  to  be  determined  after  the  test  as  discussed  above.  The  entries  for  relative 
latencies  in  Table  4.33.2.2-7  are  the  differences  between  the  latency  of  each  type  of  data  received 
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at  the  WSIC  simulation  and  the  latency  of  the  same  type  received  at  the  SIMLAB  simulation. 
However,  note  that  in  this  case  the  relative  target  entity  state  data  latency  was  only  for  data 
transmitted  from  the  WSIC  to  the  SIMLAB  (these  latencies  are  negative  in  Table  4.33.2.2-7 
because  the  target  data  was  first  generated  at  the  WSIC  and  then  received  at  the  SIMLAB,  so  that 
the  target  data  had  a  higher  latency  at  the  SIMLAB). 


Table  4.33.2.2-7.  Differences  Between  Launch  Conditions  from  WSIC  Simulation  and 
Launch  Conditions  from  SIMLAB  Simulation  (11/19/96) 


Relative  Latencies  (ms) 

Run  # 

Range  (ft) 

Sh  Alt  (ft) 

Aspect 

n 

Lead  (°) 

Launch 

Shooter 

Target 

9 

7.3 

-7.3 

1.16 

-0.25 

167 

68 

-51 

10 

25.4 

-3.8 

2.50 

-038 

68 

68 

-193 

12 

28.6  1 

-6.4 

1.16 

-0.60 

163 

63 

-80 

18 

54.0 

-5.9 

0.71 

-0.53 

157 

56 

-29 

19 

36.3 

-9.9 

1.36 

-0.54 

157 

5 

-119 

22 

70.9 

-5.9 

1.77 

-0.57 

229 

77 

-80 

Mean 

37.1 

-6.5 

1.44 

-0.48 

156.8 

56.2 

-92.0 

Std  Dev 

22.5 

2.0 

0.62 

0.14 

51.5 

26.0 

58.1 

Table  4.33.2.2-7  shows  very  good  agreement  for  all  launch  conditions.  Note  that  the  target 
entity  state  data  latency  between  the  WSIC  and  the  SIMLAB  at  launch  was  comparable  to  the 
mean  value  for  data  passed  after  launch  (Table  4.3.3.1-13),  but  the  relative  latencies  of  the  launch 
indication  were  much  larger.  Nevertheless,  even  with  relative  latencies  up  to  about  150  msec,  the 
agreement  in  the  launch  conditions  was  still  very  good. 

Comparisons  Between  Logged  PDU  Data  .  The  differences  between  the  launch  conditions  from 
PDU  data  logged  at  various  locations  were  computed  and  were  found  to  be  in  excellent 
agreement,  as  in  the  V&V  Mission  (Table  433.2.2-4). 

STMLAB  Simulation  Data  Compared  to  “No  Latency”  PDU  Data  .  Finally,  the  differences 
between  the  launch  conditions  from  the  SIMLAB  simulation  and  those  from  entity  state  PDUs 
according  to  PUTT  time  were  computed  and  are  given  in  Table  433.2.2-8.  The  shooter  and 
target  entity  state  PDUs  were  identified  which  had  the  same  PDU  time  as  the  Fire  (Missile)  PDU. 
Note  that  these  PDUs  were  being  created  at  the  WSSF  and  WSIC,  respectively,  at  the  time  the 
Fire  (Missile)  PDU  was  created  at  the  SIMLAB  and  had  not  yet  come  together  at  any  one 
location  for  direct  observation.  Instead,  computing  the  launch  conditions  from  PDUs  using  the 
PDU  time  represented  an  approximation  of  the  “no  latency”  case  which  reflected  where  the 
shooter  and  target  actually  were  at  the  instant  of  missile  launch.  The  latencies  in  Table  433.2.2- 
8  are  for  the  shooter  entity  state  data  between  the  WSSF  and  the  SIMLAB  and  for  the  target 
entity  state  data  between  the  WSIC  and  the  SIMLAB. 

Table  433.2.2-8  shows  very  good  agreement  for  all  launch  conditions,  even  with  latencies  up  to 
about  100  msec. 
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Table  4.3.3.2.2-8.  Differences  Between  Launch  Conditions  from  WSSF  Simulation  and 
Launch  Conditions  from  PDUs  Using  PDU  Time  (11/19/96) 


SIMLAB  Latencies  (ms) 

Run  # 

Range (ft) 

Sh  Alt  (ft) 

GniM 

Lead  (°) 

Shooter 

Target 

9 

74.6 

6.7 

-0.47 

-0.24 

78 

51 

|  10 

69.4 

3.3 

-1.76 

-0.13 

79 

193 

12 

57.3 

5.3 

-0.52 

-0.10 

59 

80 

18 

97.0 

8.2 

-0.18 

0 

118 

29 

19 

68.2 

8.1 

-0.89 

-0.01 

117 

119 

22 

62.4 

3.9 

-0.69 

-0.61 

108 

80 

Mean 

71.5 

5.9 

-0.75 

-0.18 

93.2 

92.0 

Std  Dev 

13.9 

2.1 

0.55 

0.23 

24.5 

58.1 

CONCLUSION .  The  small  and  stable  latencies  achieved  in  the  Parametric  Study  Mission  resulted 
in  good  agreement  in  the  launch  conditions  determined  by  the  various  simulations  (within  10%  of 
shot  box  tolerances).  This  was  hue  even  though  latencies  as  large  as  about  200  msec  were 
observed  between  simulations  for  some  of  the  data.  In  particular,  the  shooter  and  target 
simulations  were  in  sufficient  agreement  to  allow  this  ADS  architecture  to  be  used  for  pre-launch, 
closed-loop  interactions,  such  as  rehearsal  and  refinement  of  live  engagement  scenarios. 

4.3.3.2.3  Terminal  Engagement  Conditions  Differences  Between  Nodes 

The  terminal  range  was  the  range  between  the  missile  and  the  target  when  the  SIMLAB 
simulation  stopped  the  missile  flyout.  Typically,  the  missile  had  a  time  to  go  of  100  msec  at  this 
time.  The  terminal  range  was  not  the  miss  distance.  Rather,  the  miss  distance  was  estimated  in 
the  SIMLAB  simulation  by  dead  reckoning  the  missile  and  target  velocities  from  the  terminal 
range  until  the  distance  of  closest  approach  was  obtained.  The  last  missile  entity  state  PDU  was 
generated  when  the  missile  reached  the  terminal  range  and  the  missile  flyout  ended,  and  this  PDU 
was  transmitted  repeatedly  from  the  SIMLAB  for  about  one  second  after  the  end  of  the  missile 
flyout. 

The  terminal  range  was  computed  from  logged  PDU  data  using  the  following  steps: 

-  The  first  repeating  missile  entity  state  PDU  at  a  node  was  identified. 

—  This  indicated  the  end  of  the  missile  flyout. 

-  Initially,  the  Detonation  PDU  was  to  be  used.  However,  this  was  found  to  be 
generated  up  to  one  second  after  the  end  of  the  missile  flyout. 

-  The  next  logged  target  entity  state  PDU  was  identified. 

-  The  terminal  range  between  the  missile  and  target  was  computed  using  the  PDU  positional 
data. 
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Initially,  the  terminal  range  computed  from  the  entity  state  PDU  data  was  to  be  compared  to  the 
terminal  range  determined  by  the  SIMLAB  simulation.  However,  two  problems  with  the 
SIMLAB  simulation  prevented  this  direct  comparison:  the  error  in  initializing  the  target  position 
in  the  SIMLAB  simulation  reference  frame  and  the  latitude  divergence  problem  (Section  4. 1.1.3). 
Instead,  the  terminal  range  derived  from  entity  state  PDUs  logged  at  the  WSIC  were  compared  to 
those  derived  from  entity  state  PDUs  logged  at  the  SIMLAB.  This  comparison  should  provide 
some  insight  into  how  well  the  missile  and  target  simulations  might  agree  on  miss  distance.  The 
results  from  the  Parametric  Study  Mission  are  given  in  Table  43.3.2.2-9.  The  relative  latency  of 
the  missile  entity  state  PDUs  (difference  between  latency  at  WSIC  and  latency  at  SIMLAB)  is 
given  in  the  table  as  a  representative  latency. 


Table  4.3.3.2.2-9.  Differences  Between  Terminal  Range  from  PDUs  Logged  at  WSIC  and 
Terminal  Range  from  PDUs  Logged  at  SIMLAB  (11/19/96) 


Run  # 

Terminal  Range 
Difference  (ft) 

Relative  Missile 
Latency  (ms) 

9 

32.0 

19 

10 

0 

18 

12 

37.2 

17 

18 

0 

15 

19 

0 

16 

22 

35.6 

18 

Table  4.33.2.2-9  shows  the  following: 

-  The  terminal  ranges  from  PDU  data  agreed  exactly  on  half  of  the  runs.  This  resulted  when 
the  same  pair  of  missile  and  target  entity  state  PDUs  were  used  to  determine  the  terminal 
range  at  the  two  nodes.  This  could  occur  because  the  time  between  entity  state  PDUs  was 
about  50  msec,  and  the  relative  missile  latency  was  significantly  less  than  that. 

-  The  non-zero  range  differences  resulted  when  the  target  entity  state  data  from  the  WSIC 
logger  was  from  the  next  generated  PDU,  as  compared  to  the  PDU  used  from  the 
SIMLAB  logger.  Note  that  the  same  missile  entity  state  PDU  was  always  used  at  both 
nodes. 

-  The  non-zero  differ  ences  exceed  the  lethal  radius  for  the  missile,  so  that  in  those  cases,  it 
was  possible  for  the  missile  and  target  nodes  to  disagree  on  whether  or  not  the  target  was 
killed.  Hence,  terminal  engagement  results  for  a  closed-loop  interaction  between  the 
missile  and  target  would  be  invalidated. 

43.3.2.4  Synchronization  of  Simulations 

The  simulations  could  only  be  synchronized  within  the  PDU  update  time  (50  msec  during  the 
Parametric  Study  Mission).  The  simulations  ran  at  higher  rates,  but  could  only  exchange 
information  at  this  rate.  Hence,  any  simulation-to-simulation  latency  of  less  than  50  msec  would 
have  resulted  in  the  simulations  being  synchronized  to  the  degree  possible.  In  other  words, 
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latencies  less  than  50  msec  allow  a  receiving  simulation  to  be  using  the  most  current  set  of  PDU 
data  from  a  transmitting  simulation. 

The  latencies  evaluated  between  simulations  (Tables  4.3.3.1-12  and  4.3.3.1-13)  showed  that  they 
were  usually  running  one  PDU  update  time  behind  each  other  during  the  Parametric  Study 
Mission.  However,  there  were  instances  during  a  run  in  which  one  of  the  simulations  was  as 
many  as  seven  PDU  update  times  behind  the  other. 

True  synchronization  of  the  simulations  would  require  (1)  that  the  PDU  update  rate  equal  the 
simulation  frame  rate  of  the  fastest  simulation,  (2)  that  the  frames  of  the  individual  simulations  be 
synchronized  to  start  at  the  same  time  (this  also  requires  that  the  simulation  frame  rates  all  be  the 
same),  and  (3)  that  the  simulation-to-simulation  latencies  be  less  than  the  simulation  frame  time  of 
the  fastest  simulation.  Since  the  frame  time  of  the  SIMLAB  simulation  was  less  than  1  msec,  the 
PDU  update  rate  would  have  to  be  greater  than  1  kHz  and  the  latencies  would  have  to  be  less 
than  1  msec.  Such  latencies  cannot  be  achieved  with  simulation  facilities  separated  by  more  than 
100  miles. 

4.3.4  Latency  Study  Summary 

Improvements  in  the  NIU  settings  and  operations  allowed  the  latencies  to  be  greatly  reduced  by 
the  Parametric  Study  Mission.  The  latencies  exhibited  significant  sample-to-sample  variations 
during  a  run  (the  standard  deviation  of  the  latency  was  a  significant  fraction  of  the  mean  value). 
Much,  but  not  all,  of  this  variation  appeared  to  be  caused  by  the  interface  between  the  simulation 
and  the  NIU  at  the  receiving  node. 

The  latency  variations  can  distort  entity  state  data  received  by  a  simulation,  since  the  data  was 
input  into  the  simulation  at  the  rate  the  data  were  received. 

Latencies  in  the  Parametric  Study  Mission  were  small  enough  (less  than  200  msec  between 
simulations)  to  allow  the  simulations  to  agree  on  the  launch  conditions  to  within  less  than  10%  of 
the  shot  box  tolerances.  This  indicates  that  the  LSP  architecture  could  be  used  for  the  rehearsal 
and  refinement  of  launch  conditions  for  live  mission  scenarios.  However,  the  latencies  were  too 
large  to  allow  reliable  evaluation  of  the  terminal  engagement  between  the  missile  and  target. 

4.4  Test  Objective  4:  Assess  ability  of  LSP  ADS  configuration  to  support  AIM-9  testing 

The  test  objective  is  broken  into  subobjectives  as  follows. 

4.4.1  Test  Subobjective  4-1:  Assess  capability  of  ADS  network  to  provide  bandwidth  and 
connectivity  required  for  LSP  tests 

This  subobjective  is  to  assess  the  ability  of  the  LSP  ADS  network  to  support  AIM-9  testing  when 
it  is  operating. 
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4.4.1.1  Bandwidth  and  Connectivity  Test  Method 

Data  were  collected  for  this  subobjective  during  all  the  missions. 

4.4.1.2  Bandwidth  and  Connectivity  Analysis  Method 
Bandwidth  Utilization 

The  average  utilization  was  computed  for  each  trial.  Data  for  bandwidth  utilization  versus  time 
was  to  be  evaluated,  but  the  non-availability  of  the  appropriate  network  analysis  tools  prevented 
this. 

Connectivity  Loss 

The  number  of  trials  during  which  connectivity  was  lost  was  determined,  along  with  the  duration 
of  the  loss. 

4.4.1.3  Bandwidth  and  Connectivity  Results 

Typical  bandwidth  utilization  was  about  4%  on  the  T1  connecting  Point  Mugu  and  China  Lake. 
Bandwidth  utilization  on  the  WSSF  and  SIMLAB  local  area  networks  (LANs)  was  less  than  2%, 
but  about  50%  on  the  WSIC  LAN. 

There  was  no  loss  of  connectivity  during  the  LSP  missions. 

4.4.2  Test  Subobjective  4-2:  Assess  the  effects  of  ADS-induced  errors  on  LSP  test  results 
validity 

4.4.2.1  ADS-induced  Errors  Test  Method 

Data  were  collected  for  this  subobjective  during  all  the  missions. 

4.4.2.2  ADS-induced  Errors  Analysis  Method 

Data  from  the  missions  were  examined  to  determine  if  any  ADS-induced  errors  had  occurred.  The 
ADS-induced  errors  which  were  anticipated  prior  to  the  missions  included  PDUs  not  received  at 
the  appropriate  node  or  received  out  of  order,  PDUs  corrupted  during  transmission,  and  entity 
state  data  errors  introduced  during  the  coordinate  transformations  required  for  entity  state  PDUs. 
The  PDU  data  was  examined  for  other  errors  or  irregularities  not  anticipated. 

4.4.2.3  ADS-induced  Errors  Results 

Missing  PDUs 
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The  counts  of  logged  PDU  data  collected  at  the  various  locations  were  checked  to  determine  if 
the  number  of  PDUs  transmitted  by  one  node  agreed  with  the  number  of  PDUs  received  by 
another  node.  No  instances  of  missing  PDUs  were  found. 
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Out-of-Order  PDUs 


The  logged  entity  state  PDU  data  collected  at  the  various  locations  were  checked  to  determine  if 
the  PDUs  for  a  given  entity  were  received  in  the  order  they  were  created.  This  was  done  by 
arranging  the  logged  PDUs  by  log  time  and  noting  whether  or  not  the  PDU  time  increased 
monotonically.  No  instances  of  out-of-order  PDUs  were  found. 

PDUs  Corrupted  During  Transmission 

The  logged  entity  state  PDU  data  collected  at  the  various  locations  were  checked  to  determine  if 
the  entity  state  data  values  for  the  same  PDU  agreed  (PDUs  were  matched  by  PDU  time).  No 
instances  of  corrupted  PDUs  were  found. 

Coordinate  Transform  Errors 

The  net  effect  of  transforming  entity  state  data  from  the  transmitting  simulation  reference  frame  to 
the  PDU  reference  frame  and  then  to  the  receiving  simulation  reference  frame  was  evaluated  in 
Section  4.1. 1.3.  The  result  was  that  the  net  error  in  transforming  the  positional  data  was  1-2  ft., 
and  this  was  determined  to  be  quite  acceptable. 

Repeating  PDUs 

As  noted  previously,  some  of  the  entity  state  data  were  found  to  contain  repeating  PDUs.  These 
were  PDUs  which  had  the  same  PDU  time  and  entity  state  data,  but  which  were  logged  at 
receiving  locations  at  different  times  (which  implied  that  the  repeaters  were  actually  created  at 
later  times). 

During  the  V&V  Mission,  repeating  PDUs  were  noted  for  both  the  shooter  and  missile.  In  some 
cases,  the  PDUs  repeated  more  than  one  time,  resulting  in  “clumps.”  Examples  are: 

-  Run  #19  had  46  repeating  shooter  entity  state  PDUs  out  of  208  total  PDUs.  In  one  case, 
a  shooter  PDU  repeated  5  times. 

-  Run  #19  had  12  repeating  missile  entity  state  PDUs,  and  one  repeated  5  times. 

Run  #23  had  46  repeating  shooter  entity  state  PDUs  out  of  238  total  PDUs. 

-  Run  #23  had  9  repeating  missile  entity  state  PDUs,  and  one  repeated  8  times  (all  the 
repeaters  except  one  were  in  a  single  “clump”). 

During  the  Parametric  Study  Mission,  repeating  PDUs  were  also  noted  for  both  the  shooter  and 
missile.  However,  only  single  repeaters  were  noted;  there  were  no  “clumps.”  Characteristics  of 
the  repeaters  were: 

-  Only  three  of  the  six  complete  V2  runs  had  repeating  shooter  entity  state  PDUs.  The 
repeaters  tended  to  occur  well  before  missile  launch,  so  that  there  were  no  repeaters  near 
the  launch  time.  Details  are  as  follows: 
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-  Run  #9  had  60  repeating  PDUs  out  of  269  total.  There  were  none  during  the  last  5 
sec  before  launch. 

-  Run  #10  had  3  repeating  PDUs  out  of  2  53  total.  There  were  none  during  the  last  10 
sec  before  launch. 

-  Run  #18  had  25  repeating  PDUs  out  of  290  total.  There  were  none  during  the  last  10 
sec  before  launch. 

-  Only  two  of  the  six  complete  V2  runs  had  repeating  missile  entity  state  PDUs.  In  one  case 
(Run  #12)  there  was  only  one  repeater  out  of  102  total  PDUs.  In  another  case  (Run  #18), 
there  were  only  two  repeaters  out  of  1 1 1  total  PDUs. 

The  error  caused  by  repeating  entity  state  PDUs  was  causing  the  entity  position  to  either  freeze 
(if  the  receiving  node  was  not  dead  reckoning  the  entity’s  location)  or  to  jump  back  (if  the 
receiving  node  had  been  dead  reckoning  the  entity’s  location  prior  to  receiving  the  repeater).  For 
the  LSP,  the  only  entity  state  data  being  used  by  other  simulations  was  the  target’s.  Since  the 
target  entity  state  PDUs  did  not  repeat,  there  were  no  errors  in  the  simulations  caused  by 
repeating  PDUs. 

Irregular  Entity  State  Data  Update  Rates 

The  PDUs  were  not  created  at  precise  rates  by  the  NIUs.  Instead,  the  time  between  creating 
PDUs  varied,  and  in  some  cases,  there  were  significant  time  gaps  between  PDUs.  Statistics  on 
the  time  intervals  between  PDUs  are  given  in  Tables  4.4.2.3-1  and  4.4.2.3-2. 


Table  4.4.2.3-1.  Time  Interval  Between  Target  Entity  State  PDUs  (11/19/96) 


Run# 

Target  PDUs  Before  Launch  (msec) 

Target  PDUs  After  Launch  ( 

msec) 

Mean 

Std  Dev 

Min 

Max 

Mean 

Std  Dev 

Min 

Max 

9 

51.5 

18.4 

27 

152 

53.5 

34.2 

27 

360 

10 

51.6 

20.2 

26 

240 

53.8 

24.8 

26 

240 

12 

52.0 

21.4 

26 

272 

55.3 

33.0 

27 

239 

18 

50.0 

15.0 

27 

65 

56.5 

35.2 

28 

362 

19 

50.1 

13.0 

27 

90 

55.9 

32.0 

27 

299 

22 

51.7 

21.3 

26 

240 

54.1 

25.4 

26 

213 

Table  4.4.2.3-2.  Time  Interval  Between  Shooter  and  Missile  Entity  State  PDUs  (11/19/96) 


Run  # 

Shooter  PDUs  Before  Launch  (msec) 

Missile  PDUs  (msec)  1 

Mean 

Std  Dev 

Min 

Max 

Mean 

Std  Dev 

Min 

Max 

9 

50.5 

5.3 

47 

100 

60.2 

6.0 

39 

83 

10 

50.0 

2.0 

47 

53 

60.0 

3.5 

51 

69 

12 

50.5 

8.6 

47 

200 

60.2 

29.5 

9 

130 

18 

50.3 

4.2 

47 

100 

61.1 

12.6 

48 

186 

19 

50.0 

2.1 

47 

54 

60.0 

2.3 

47 

72 

22 

50.0 

2.2 

47 

54 

60.2 

3.8 

41 

79 
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The  following  is  noted  from  Tables  4.4.23- 1  and  4.4.23-2: 

-  The  creation  rate  for  the  target  entity  state  PDUs  decreased  slightly  after  missile  launch 
(from  a  mean  value  of  19.6  Hz  before  launch  to  a  mean  value  of  18.2  Hz  after  launch). 

-  There  were  significant  variations  in  the  creation  rate  of  the  target  PDUs.  Intervals  of  up 
to  360  msec  were  noted  between  PDUs. 

-  The  shooter  PDUs  had  the  most  stable  creation  rate,  although  gaps  of  up  to  200  msec 
between  PDUs  were  noted. 

-  The  missile  PDUs  had  a  low  mean  creationrate  of  16.6  Hz. 

Variations  in  the  entity  state  PDU  rate  from  a  creating  simulation  result  in  the  input  to  a  receiving 
simulation  not  being  smooth.  Long  gaps  between  PDUs  can  result  in  errors  in  the  receiving 
simulation  as  it  dead  reckons  the  position  data  (but  not  the  velocity  data). 

The  SIMLAB  integrated  the  target  velocity  to  determine  the  target  position,  and  irregular  velocity 
update  rates  resulted  in  errors  in  the  computed  target  location.  The  effect  of  the  irregular  creation 
rate  alone  on  the  computed  target  location  was  estimated  by  comparing  the  integration  of  the 
north  component  of  target  velocity  at  the  irregular  creation  rates  actually  observed  during  the 
Parametric  Study  Mission  to  the  integration  at  a  regular  20  Hz  rate.  The  result  was  that,  on  the 
average,  the  location  of  the  target  from  the  integration  of  the  irregular  velocity  update  rates  was 
about  8  ft  north  of  the  location  of  the  target  from  the  integration  of  a  regular  velocity  update  rate, 

The  irregular  creation  rate  for  the  target  entity  state  data  noted  in  Table  4.4.23-1  was  aggravated 
by  variations  in  the  latencies  between  the  WSIC  and  the  SIMLAB  simulations  and  by  a  lack  of 
synchronization  between  the  rate  at  which  the  target  entity  state  data  were  updated  at  the 
SIMLAB  NIU  and  the  rate  at  which  these  data  were  updated  in  the  SIMLAB  simulation  (so  that 
the  most  current  entity  state  data  were  not  always  input  into  the  SIMLAB  simulation).  The 
resulting  update  times  for  target  velocity  data  into  the  SIMLAB  simulation  are  given  in  Table 
4.4.23-3. 

Table  4.4.23-3.  Time  Interval  Between  Target  Velocity  Updates  in  SIMLAB  Simulation 

(11/19/96) 


Run# 

Target  Data  After  Launch  (msec) 

Mean 

Std  Dev 

Min 

Max 

9 

54.8 

41.3 

19 

446 

10 

62.5 

23.0 

38 

156 

12 

67.1 

35.0 

20 

251 

18 

82.6 

46.2 

43 

399 

19 

81.3 

41.0 

30 

300 

22 

77.6 

34.1 

41 

248 

The  following  is  noted  from  Table  4.4.23-3: 
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-  The  rates  for  updating  the  target  velocity  in  the  SIMLAB  simulation  were  significantly 
lower  than  the  creation  rates  of  the  target  PDUs  after  missile  launch  (Table  4.4.2.3-1). 
The  average  rate  of  creating  the  PDUs  was  18.2  Hz,  but  the  mean  update  rate  in  the 
SIMLAB  simulation  was  as  low  as  12.1  Hz  for  Run  #19. 

-  The  low  velocity  update  rates  contributed  to  the  latitude  divergence  problem  discussed  in 
Section  4.1. 1.3.  The  actual  times  between  velocity  updates  were  used  to  estimate  the 
SIMLAB  integration  errors  in  Table  4. 1.1. 3-5. 

The  latitude  divergence  discussed  in  Section  4. 1.1. 3  compared  the  SIMLAB-computed 
latitude  of  the  target  with  the  target  latitude  from  PDU  data  input  into  the  SIMLAB 
simulation.  When  the  entity  state  PDUs  were  not  updated  at  the  SIMLAB,  the  target 
position  (including  latitude)  were  dead  reckoned.  Dead  reckoning  the  target  latitude 
resulted  in  the  target  position  being  located  north  of  its  actual  location  (the  target  north 
component  of  velocity  was  constantly  decreasing  with  time,  so  that  too  large  of  a  north 
velocity  was  used  in  the  dead  reckoning). 

--  The  effect  of  the  irregular  PDU  update  rates  on  the  target  latitude  is  illustr  ated  in 
Figure  4.1. 1.3-1.  Note  that  the  solid  curve  for  the  target  latitude  input  into  the 
SIMLAB  simulation  is  not  smooth,  but  displays  some  jagged  features.  This  was  due 
to  the  dead  reckoning  of  the  target  latitude  between  PDU  updates.  When  the  latitude 
was  finally  updated,  a  discontinuous  adjustment  resulted. 

PDU  Time  Stamp  Errors 

Occasionally,  shooter  entity  state  PDUs  were  logged  which  had  the  same  PDU  time,  but  different 
entity  state  data.  Since  the  resulting  PDU  had  an  invalid  PDU  time,  any  latency  determination 
based  on  the  PDU  time  was  also  invalid.  (Note  that  the  simulations  did  not  use  PDU  time;  as  they 
received  data,  they  used  it  without  regard  to  when  the  data  were  created.)  However,  this  type  of 
error  was  very  infrequent  (only  detected  two  or  three  times  during  all  of  the  LSP  runs). 

4.4.3  Test  Subobjective  4-3:  Assess  adequacy  of  standard  data  protocols  for  LSP  test 

4.4.3.1  Data  Protocol  Adequacy  Test  Method 

Data  were  collected  for  this  subobjective  during  all  the  missions. 

4.4.3.2  Data  Protocol  Adequacy  Analysis  Method 

DIS  PDUs  were  identified  for  use  in  the  LSP  and  were  deemed  to  be  adequate  for  all  data 
exchanges  with  the  exception  of  the  SMS  data  (for  which  no  standard  protocol  exists). 

Any  bandwidth  utilization  problems  identified  under  Subobjective  4-1  and  any  ADS-induced 
errors  identified  under  Subobjective  4-2  were  to  be  analyzed  to  determine  if  the  structure,  format, 
or  processing  of  the  PDUs  was  the  cause  of  the  error. 

-  The  bandwidth  required  for  the  data  in  the  high-f  idelity  simulation  output  format  was  to  be 
compared  to  the  same  data  in  the  PDU  format.  The  data  rate  output  directly  from  the 
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high-fidelity  simulations  was  to  be  determined  from  data  logged  on  the  simulation 
computers.  The  data  rate  output  from  the  NIU  at  each  node  was  to  be  determined  from 
file  PDU  data  loggers.  The  differences  were  to  be  computed  at  several  sample  times 
during  periods  when  bandwidth  utilization  problems  occurred. 

-  High-fidelity-data-to-PDU  conversion  errors  and  PDU-to-high-fidelity-da  ta  conversion 
errors  analyzed  under  Subobjective  4-2  were  reviewed  to  determine  if  the  source  of  the 
error  was  the  reference  frame  conversions  required  for  entity  state  PDUs. 

4.4.3.2  Data  Protocol  Adequacy  Results 

Adequate  tools  were  not  available  to  identify  specific  periods  of  high  bandwidth  utilization. 
However,  the  typical  bandwidth  utilization  on  the  T1  connecting  Point  Mugu  and  China  Lake  was 
relatively  low  (~4%).  This  indicated  that  use  of  the  PDUs  did  not  result  in  bandwidth  utilization 
problems. 

The  coordinate  transform  errors  associated  with  the  use  of  entity  state  PDUs  were  small  (1-2  ft) 
and  deemed  to  be  quite  acceptable. 

Data  PDUs  were  used  to  transmit  test  control  display  data  from  the  WSSF  and  WSIC  to  the 
TCAC.  The  data  PDUs  did  an  adequate  job  of  transmitting  these  data. 

CONCLUSION.  The  DIS  PDUs  used  in  the  LSP  were  adequate  for  the  required  data  exchanges. 

4.4.4  Test  Subobjective  4-4:  Assess  reliability,  availability,  and  maintainability  of  ADS 
network 

Reliability,  availability,  and  maintainability  (RAM)  data  were  gathered  for  all  components  that 
made  up  the  LSP  ADS  configuration  at  each  node  as  well  as  the  linking  network.  The  data 
gathered  at  each  of  the  four  nodes,  (BMIC/TCAC,  WSIC,  WSSF,  and  SIMLAB)  consisted  of  all 
network-related  hardware  and  software  failures  which  occurred  during  the  test  events.  Failures  of 
the  high-fidelity  simulations  themselves  were  not  included. 

4.4.4.1  RAM  Test  Method 

Data,  as  shown  in  Table  4.4.4. 1-1,  were  collected  for  this  subobjective  during  all  the  missions. 
Operator  logs,  maintenance  logs,  and  test  director  logs  were  the  major  sources  of  data. 
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Table  4.4.4.1-1.  Reliability,  Availability,  and  Maintainability  Data  Requirements 


Data  Requirement 

Definition 

Machine  Name 

Name  of  hardware  that  failed 

Program/Device  Name 

Name  of  software  with  fault  and  device  it  was  running  on 

Failure  Type 

Either  hardware  failure  or  software  fault 

Mission  Start  or 

Last  Time  Up 

Either  time  when  mission  started  (if  first  failure  during  mission)  or  last 
time  when  entire  ADS  network  was  available  (if  not  first  failure) 

Time  of  Failure  (TF) 

Time  during  mission  when  failure  occurred 

Time  when  repair  of  hardware  failure  or  software  fault  is  completed 

Time  when  network  is  available  to  resume  testing  ! 

4.4.4.2  RAM  Analysis  Method 

The  RAM  of  the  LSP  ADS  configuration  was  assessed.  Although  component  RAM  data  were 
gathered,  only  overall  system  and  linking  network  RAM  are  reported  due  to  the  limited  test  time 
available.  Individual  RAM  data  are  not  reported  for  the  three  simulators  used  in  this  test.  The 
entire  ADS  configuration  and  network,  not  the  simulators,  were  the  focus  of  this  study. 


The  data  indicated  in  Table  4.4.4.1-1  were  to  be  used  to  calculate:  Availability  (  Ao),  Reliability 
expressed  as  Mean  Time  Between  Operational  Mission  Failures  (MTBOMF),  and  Maintainability 
expressed  as  Mean  Corrective  Maintenance  Time  for  Operational  Mission  Failures 
(MCMTOMF).  Both  MTBOMF  and  MCMTOMF  were  to  be  broken  down  into  both  hardware 
and  software  failures  and  repairs. 

Availability  The  parameter  for  addressing  availability  (\0)  were  calculated  as: 


A0 


Up  Time 

Up  Time  +  Down  Time 


(7) 


-  Up  Time  was  that  time  duration  when  the  system  was  considered  to  be  ready  for  use  and 
was  either  operating  or  in  standby  (in  an  up  status). 

—  Each  increment  of  Up  Time  was  computed  from  the  difference  between  the  last  Time 
Up  (TU)  and  the  next  Time  of  Failure  (TF). 

-  The  increments  of  Up  Time  were  added  to  give  the  total  Up  Time  for  (1)  each  mission 
and  for  (2)  all  LSP  missions. 

-  Down  Time  was  that  time  duration  when  the  system  was  down  for  repair  of  operational 
mission  hardware  failures  and/or  for  restoration  from  operational  mission  software  faults. 

-  Down  Time  includes  time  to  detect  failure,  investigation  time,  reporting  time,  time 
awaiting  repair,  fault- fix  time,  restart  time,  and  checkout  time. 

—  Each  increment  of  Down  Time  was  computed  from  the  difference  between  the  next 
Time  Up  (TU)  and  the  last  Time  of  Failure  (TF). 

Reliability  The  parameters  for  addressing  reliability  (MTBOMF  system,  MTBOMF hw,  and 
MTBOMFsw)  were  to  be  calculated  as: 
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MTBOMF 


_ Total  System  Operating  Time 

system  Number  of  Operational  Mission  Failures  /  Faults 


An  operational  mission  failure/fault  was  one  which  precluded  successful  completion  of  a 
mission  and  included  a  hardware  failure  or  a  software  fault. 

System  Operating  Time  was  to  include  all  time  that  the  system  was  operating  or  in 
standby.  This  was  the  same  as  the  total  Up  Time  during  a  mission. 


MTBOMF  HW  = 


Total  System  Operating  Time _ 

Number  of  Operational  Mission  Hardware  Failures 


An  operational  missio  n  hardware  failure  was  one  which  precluded  successful  completion 
of  a  mission. 


MTBOMF  sw  = 


Total  System  Operating  Time _ 

Number  of  Operational  Mission  Software  Faults 


An  operational  mission  software  fault  was  one  which  precluded  successful  completion  of  a 


mission. 


Maintainability .  The  parameters  for  addressing  maintainability  (MCMTOMF  Hw  and 
MCMTOMFsw)  were  to  be  calculated  as: 

Total  Elapsed  Time  to  Repair  Hardware  Failures  .  . 

MCMTOMF  HW  = - — - 77; - -r-  TTT  T - ^ -  0 1) 

Total  Number  of  Operational  Hardware  Failures 

-  Repair  time  for  hardware  failures  included  maintenance  preparation,  fault  location  and 
isolation,  fault  correction,  adjustment  and  calibration. 

Total  Elapsed  Time  to  Repair  Software  Faults 
MCMTOMF  sw  rpQtaj  dumber  of  Operational  Software  Faults 

-  Repair  time  for  software  faults  was  the  time  needed  to  restore  all  system  processes, 
functions,  files,  and  data  bases  to  a  useful  state. 


4A.4.3  RAM  Analysis  Results 

Availahilitv .  A  summaiy  of  the  periods  of  Up  Time  and  Down  Time  for  linked  testing  periods  is 
given  in  Table  4.4.4.3-1.  This  table  also  summarizes  results  from  the  linked  laboratory  testing 
periods  which  were  used  to  prepare  for  the  missions. 
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Table  4.4.4.3-1.  Summary  of  Periods  of  Up  Time  and  Down  Time  for  All  Linked  Testing 


Date 

Total  Mission  Time 

Up  Time 

Down  Time 

27  Aug  96  (Rehearsal) 

4  hr  14  min 

4  hr  14  min 

None 

28  Aug  96  (Rehearsal) 

4  hr  13  min 

4  hr  6  min 

7  min 

22  Oct  96  (Lab  Time) 

1  hr  52  min 

1  hr  40  min 

12  min 

29  Oct  96  (Mission) 

3  hr  55  min 

3  hr  13  min 

42  min 

1  hr  43  min 

1  hr  43  min 

None 

19  Nov  96  (Mission) 

3  hr  45  min 

1  hr  54  min 

1  hr  51  min 

Totals 

19  hr  42  min 

16  hr  50  min 

2  hr  52  min 

As  Table  4.4.4.3-1  shows,  the  total  Up  Time  for  the  mission  rehearsal,  the  two  missions  and  the 
two  formal  linked  laboratory  periods  was  16  hours  and  50  minutes,  and  the  total  Down  Time  was 
2  hours  and  52  minutes.  Using  these  values  in  Equation  7  gives  an  average  availability  of  85.4%. 


Reliability.  There  were  no  operational  mission  failures/faults.  Hence,  the  reliability  parameters 
(MTBOMF  system,  MTBOMFhw,  and  MTBOMF  sw)  could  not  be  calculated.  However,  it  can  be 
noted  that  these  mean  times  between  operational  mission  failures  must  exceed  the  total  mission 
time  of  19  hours  and  12  minutes. 

Maintainability .  A  summary  of  die  times  to  repair  the  hardware  failures  is  given  in  Table  4.4.4.3- 
2,  and  a  summary  of  the  times  to  repair  the  software  faults  is  given  in  Table  4.4.4.3-3.  Note  that 
these  failures/faults  all  resulted  in  Down  Time,  so  that  the  total  repair  time  equals  the  Down  Time. 


Table  4.4.4.3-2.  Summary  of  Repair  Times  for  Hardware  Failures 


Date 

Hardware  Failure  Details 

29  Oct  96  (Mission) 

Power  Loss  in  TCAC 

15  min 

19  Nov  96  (Mission) 

WSIC  blower  loss 

24  min 

19  Nov  96  (Mission) 

AIM-9  Seeker  Failure  (Connector  Problem) 

58  min 

Total 

3  hardware  failures 

1  hr  37  min 

As  Table  4.4.4.3-2  shows,  the  total  time  to  repair  three  hardware  failures  was  1  hr  37  min.  Using 
these  values  in  Equation  1 1  gives  MCMTOMftW  =  32.3  min. 
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Table  4.4.4.3-3.  Summary  of  Repair  Times  for  Software  Faults 


Date 

Software  Fault  Details 

28  Aug  96  (Rehearsal) 

SIMLAB  NIU  required  reset 

4  min 

28  Aug  96  (Rehearsal) 

SIMLAB  NIU  required  reset 

3  min 

22  Oct  96  (Lab  Time) 

WSSF  NIU  required  reset 

2  min 

22  Oct  96  (Lab  Time) 

MNS-1  (SMS  link)  required  reset 

4  min 

SIMLAB  NIU  required  reset 

5  min 

22  Oct  96  (Lab  Time) 

SIMLAB  NIU  required  reset 

1  min 

29  Oct  96  (Mission) 

SIMLAB  NIU  required  reset 

1  min 

29  Oct  96  (Mission) 

WSIC  simulation  data  logging  system  crash 

16  min 

29  Oct  96  (Mission) 

SIMLAB  NIU  required  reset 

1  min 

29  Oct  96  (Mission) 

SIMLAB  simulation  computer  crash  (over  heat) 

5  min 

29  Oct  96  (Mission) 

WSIC  simulation  required  reset 

4  min 

19  Nov  96  (Mission) 

WSIC  simulation  required  reset 

16  min 

19  Nov  96  (Mission) 

WSIC  simulation  required  reset 

4  min 

19  Nov  96  (Mission) 

SIMLAB  simulation  computer  crash 

8  min 

19  Nov  96  (Mission) 

SIMLAB  simulation  computer  crash 

1  min 

Total 

15  software  faults 

1  hr  15  min 

As  Table  4.4.3-3  shows,  the  total  time  to  repair  15  software  faults  was  1  hr  15  min.  Using  these 
values  in  Equation  12  gives  MCMTOMEw  =  5.0  min. 


4.4.5  Test  Subobjective  4-5:  Assess  capability  for  centralized  test  control  and  monitoring 

4.4.5.1  Test  Control  Test  Method 

Data  were  collected  for  this  subobjective  during  all  the  missions. 

4.4.5.2  Test  Control  Analysis  Method 

The  ability  to  exercise  the  control  and  monitoring  functions  were  assessed.  The  following  were 

analyzed  during  each  mission  and  rehearsal: 

-  Ability  to  coordinate  the  start  and  stop  of  each  simulation  run.  Could  the  start  and  stop  of 
all  simulation  facilities  be  coordinated  with  sufficient  synchronization? 

Ability  to  maintain  communications  between  all  nodes.  Were  comm  unications  to  any 
facility  lost  at  any  time? 

-  Ability  to  monitor  status  of  each  simulation  facility  during  mission.  Could  the  operational 
status  of  each  facility  be  monitored  continuously  and  in  real-time  during  each  mission? 

-  Ability  to  monitor  analog  and  discrete  data  channels  from  simulation  facilities.  Were  the 
data  from  these  channels  sufficient  to  remotely  monitor  simulation  performance? 

-  Ability  to  verify  profile  execution  and  data  collection  between  trials.  Were  the  quick-look 
results  sufficient  to  verify  that  the  planned  profile  was  executed  within  acceptable 
tolerances?  Were  the  quick-look  results  sufficient  to  verify  that  all  required  data  were 
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obtained?  Were  the  quick-look  results  available  in  a  timely  manner  (within  5  minutes  of 
trial  completion)? 

4.4.5.3  Test  Control  Analysis  Results 
4.4.5.3.1  Control  Procedures 

Test  control  procedures  were  designed  for  simplicity  and  decentralization  of  control.  The  design 
of  the  test  required  the  shooting  aircraft  to  achieve  a  certain  geometry  where  altitude,  airspeed, 
range  to  target,  target  aspect,  missile  pointing  angle  and  target  acceleration  were  simultaneously 
achieved.  The  only  way  these  conditions  could  be  repeated  consistently  was  if  the  shooter  pilot 
controlled  the  entire  intercept.  To  reduce  the  number  of  variables  he  had  to  deal  with,  the  shooter 
pilot  was  “unfrozen”  at  a  specified  distance  aft  of  the  target  with  approximately  a  half  mile  of 
lateral  separation  (see  Fig.  4.4.5.3.1-1).  After  both  simulators  were  reset  and  frozen,  the  target 
simulation  was  started.  The  shooter  watched  his  HUD  as  the  target  range  opened.  When  he  saw 
the  range  to  target  he  wanted,  he  unfroze  his  own  simulation.  This  procedure  was  used  instead  of 
simultaneously  unfreezing  both  simulations  since  the  time  required  to  unfreeze  the  target 
simulation  took  from  less  than  a  second  to  up  to  3  seconds.  This  procedure  permitted  the  shooter 
to  achieve  a  consistent  starting  geometry  without  having  to  make  a  difficult  and  time  consuming 
speed  adjustment  to  achieve  proper  range.  The  initial  altitude  difference  and  lateral  separation 
were  determined  by  trial  and  error. 

When  the  test  controller  saw  both  aircraft  “flying,”  he  called  for  the  target  to  turn  and  cross  in 
front  of  the  shooter  (see  Fig.  4.4.5.3.1-1).  The  target  then  flew  a  turn  at  constant  speed,  altitude 
and  turn  acceleration.  This  flight  path,  combined  with  the  initial  conditions  of  each  simulator, 
allowed  the  shooter  to  achieve  the  desired  range  and  aspect  parameters  with  little  maneuvering. 
This  was  critical  since  too  much  rolling  by  the  F/A-18  pushed  the  SIMLAB  HWTL  missile  beyond 
the  limits  of  the  Carco  table. 

Once  the  test  controller  had  started  the  target  turning,  achieving  the  desired  geometry  was  totally 
under  the  control  of  the  F/A-18  pilot.  After  the  missile  finished  its  flyout  the  test  controller 
terminated  the  run  and  released  the  simulators  to  reset  following  any  quick  look  procedures  they 
needed  to  perform. 

All  analysts  were  located  in  the  SIMLAB  where  the  data  from  the  missile  were  presented.  The 
only  communication  required  from  the  SIMLAB  was  a  statement  on  whether  the  run  was 
acceptable  or  not. 

This  control  approach  resulted  in  minimal  communications  on  the  unclassified  command  voice 
circuit. 


* 
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Latitude 


Figure  4.4.5.3.2-1.  Pre-Launch  Shooter  and  Target  Trajectories  (Run  #9  on  11/19/96) 

“God’s-Eye”  View 


4.4.S.3.2  Test  Monitoring  Displays 

Two  separate  displays  were  developed  to  permit  effective  control  of  the  tests  and  to  evaluate  how 
well  each  intercept  was  performed. 


-  The  first  display  was  a  two-dimensional  (“God’s-eye”)  stealth  viewer.  Entity  state  PDUs 
from  the  two  aircraft  and  the  missile,  after  it  was  fired,  were  displayed  on  a  screen 
showing  an  outline  of  the  China  Lake  restricted  airspace.  Since  both  players  were  virtual, 
the  exact  location  did  not  matter.  Each  entity  was  represented  by  a  friendly  or  hostile 
symbol  with  an  aspect  vector  whose  direction  represented  the  entity  heading  and  whose 
length  was  proportional  to  the  speed.  Each  entity  created  a  “trail”  of  points  representing 
its  flight  path.  Because  the  LSP  was  attempting  to  replicate  a  single  profile,  the  ground 
track  for  each  run  essentially  overlaid  the  tracks  of  the  previous  runs.  This  provided  a 
useful  capability  for  making  a  first  cut  evaluation  of  how  well  the  intercept  and  flyout 
replicated  the  desired  trajectories. 

The  other  “discretes”  display  showed  various  parameters  which  defined  the  “shot  box.” 
Target  altitude,  mach  number,  and  turning  acceleration,  long  with  shooter  altitude  and 
mach  number  were  displayed  in  strip  chart  format  with  a  total  of  16  seconds  of  history 
trace  visible  at  a  time.  A  horizontal  blue  line  was  displayed  continuously  for  the  desired 
value  for  each  parameter  throughout  the  period  when  the  instantaneous  value  was  being 
displayed.  A  digital  readout  for  each  parameter  was  also  provided.  When  each  parameter 
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fell  within  the  desired  range  of  values  which  defined  the  shot  box,  the  digital  readout 
turned  from  white  to  green. 

The  speed  with  which  the  shooter  was  closing  on  the  target  (  Vc)  was  also  displayed  in  this 
manner;  however,  this  parameter  turned  out  not  to  be  useful. 

The  radar  mode  and  status  of  master  arm  power  and  the  trigger  were  listed  in  gray  and  turned 
green  when  selected  or  the  trigger  was  squeezed. 

Range  to  target  was  shown  to  one  decimal  place  of  precision.  The  numbers  were  green  when 
within  the  desired  range. 

The  true  heading  of  each  aircraft  was  displayed  in  digital  format. 

Two  circular  displays  showed  (a)  where  the  radar  was  pointing  relative  to  the  nose  of  the  aircraft 
both  in  azimuth  and  elevation  and  (b)  the  angle  between  the  nose  of  the  target  and  the  lme-of- 
sight  to  the  shooter  (i.e.,  the  target  aspect  angle). 

The  radar  pointing  angle  was  used  as  a  reasonably  close  measure  of  how  far  off  boresight  the 
missile  seeker  was  pointing  since  the  seeker  was  slaved  to  the  radar  line-of-sight  at  launch. 

Each  display  turned  from  white  to  green  when  the  particular  parameter  met  acceptable  launch 
conditions.  This  provided  a  very  simple  and  accurate  method  of  determining  whether  the 
particular  run  met  the  desired  parameters.  However,  the  final  decision  on  whether  to  count  the 
run  as  “in  the  shot  box”  rested  with  the  analysts  in  the  SIMLAB. 

4.4.5.3.3  Test  Control  Evaluation 

Each  bullet  in  Section  4.4.5.2  is  addressed  below. 

-  Ability  to  coordinate  the  start  and  stop  of  each  simulation  run.  Successful.  Could 
the  start  and  stop  of  all  simulation  facilities  be  coordinated  with  sufficient 
synchronization?  Yes,  this  was  straight  forward.  Each  site  supervisor  reported  ready  on 
the  command  voice  net.  The  Test  Controller  counted  down  to  a  “start  run”  call.  Since  all 
the  site  and  laboratory  supervisors  and  both  pilots  were  on  the  command  voice  circuit, 
close  coordination  was  achieved.  A  verbal  call  to  the  Network  Coordinator,  who  was 
sitting  next  to  the  Test  Controller  in  the  TCAC,  to  start  and  stop  the  data  recorders 
ensured  all  recorders  were  running  before  the  pilots  started  flying  and  not  turned  off  until 
after  the  missile  had  completed  its  flyout. 

-  Ability  to  maintain  communications  between  all  nodes.  Successful.  Were 
communications  to  any  facility  lost  at  any  time?  While  it  would  have  been  preferable 
to  have  secure  communications  between  all  participants  on  the  command  net,  the  test  was 
successfully  accomplished  at  the  unclassified  level.  A  lack  of  phone  lines  in  some  areas  of 
the  laboratories  required  using  splitters  with  die  resultant  decrease  in  audio  levels.  This 
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was  an  inconvenience;  however,  due  to  the  simple  nature  of  the  procedures  this  was  not  a 
problem.  For  more  complex  tests  a  more  robust,  reliable  (and  expensive!) 
communications  system  would  be  required. 

Ability  to  monitor  status  of  each  simulation  facility  during  mission.  Successful. 
Could  the  operational  status  of  each  facility  be  monitored  continuously  and  in  real¬ 
time  during  each  mission?  Yes.  The  operational  status  of  each  facility  was  apparent 
from  the  communications  on  the  command  net.  There  was  no  need  for  remote  monitoring 
of  the  various  simulation  computers  or  other  components.  The  Network  Coordinator  was 
able  to  monitor  the  status  of  the  routers  and  PDU  loggers  remotely  from  the  TCAC  using 
NetSnoop  and  NetVis  software.  It  was  also  evident  fairly  quickly  on  the  stealth  viewer  if 
one  of  the  entities  was  not  putting  out  PDUs. 

Ability  to  monitor  analog  and  discrete  data  channels  from  simulation  facilities. 
Successful.  Were  the  data  from  these  channels  sufficient  to  remotely  monitor 
simulation  performance?  The  display  described  previously  provided  sufficient 
information  for  a  test  controller  (who  was  a  pilot)  to  visualize  the  progress  of  the 
intercept.  It  was  designed  to  provide  the  essential  elements  of  information  which  were 
displayed  in  the  aircraft  HUD.  During  one  of  the  rehearsal  missions  the  Test  Controller 
happened  to  be  located  in  the  SIMLAB  with  a  HUD  repeater,  but  no  stealth  viewer.  This 
configuration  worked  well  also,  even  without  any  direct  readout  of  parameters  from  the 
target  aircraft.  This  display  was  not  designed  to  monitor  any  performance  parameters 
internal  to  each  simulation.  This  was  done  by  the  laboratory  supervisor  at  each  site. 

Ability  to  verify  profile  execution  and  data  collection  between  trials.  Successful. 
Were  the  quick-look  results  sufficient  to  verify  that  the  planned  profile  was  executed 
within  acceptable  tolerances?  Yes.  The  two-dimensional  viewer  permitted  immediate 
evaluation  of  whether  or  not  the  missile  guided.  As  mentioned  above,  the  trail  feature 
provided  a  template  of  the  proper  trajectories.  In  the  case  of  the  SIMLAB,  the  physical 
limitations  of  the  Carco  table  were  such  that  either  the  missile  guided  to  intercept  or  the 
table  mirrors  exceeded  a  mechanical  limit  during  flyout  and  the  missile  “stopped”  flying 
while  pointing  on  a  heading  which  was  clearly  inappropriate.  The  parameter  display  was 
sufficient  to  evaluate  whether  all  the  parameters  of  the  shot  box  were  met.  Again,  the 
analysts  in  the  SIMLAB  had  the  final  say  on  whether  the  run  was  within  the  shot  box. 
Were  the  quick-look  results  sufficient  to  verify  that  all  required  data  were  obtained? 
Yes.  Following  each  run,  the  F/A-18  and  SIMLAB  supervisors  recorded  launch  and 
flyout  data  from  displays  in  the  cockpit  or  laboratory.  The  simulations  were  not  reset  for 
the  next  run  until  all  data  had  been  recorded.  Were  the  quick-look  results  available  in  a 
timely  manner  (within  5  minutes  of  trial  completion)?  Yes.  The  recording  process 
took  two  or  three  minutes  at  the  most. 
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4.5  Analysis  Summary 

Test  Objective  1:  Assess  the  validity  of  AIM-9  data  obtained  in  the  LSP  ADS  configuration 

-  Verification 

—  The  simulation  facilities  were  properly  linked. 

-  The  errors  in  transforming  from  the  raw  simulation  positional  data  to  entity  state  PDU 
data  were  1-2  ft,  and  these  errors  were  acceptable. 

—  There  were  no  errors  in  transforming  velocity  and  orientation  data. 

~  There  were  no  PDU  transmission  errors. 

~  There  were  several  errors  in  the  target  positional  data  presented  to  the  SIMLAB 
simulation. 

—  Random  latency  variations  introduced  uncertainty  in  the  target  position.  The 
random  nature  of  these  variations  prevents  the  future  implementation  of  a 
deterministic  real-time  correction  for  all  latency  effects. 

...  The  target  latitude  determined  by  the  SIMLAB  simulation  diverged  from  the 
WSIC  value  during  die  missile  flyout.  This  appeared  to  be  fixable  by  using  a  more 
sophisticated  target  velocity  integration  technique  and  by  using  higher  velocity 
update  rates. 

...  The  target  representation  in  the  SIMLAB  simulation  coordinate  frame  was  wrong 
due  to  an  error  in  the  coordinate  transformation.  This  was  subsequently  fixed. 

-  Validation 

-  The  SIMLAB  standalone  simulations  of  the  LPN-15  engagement  were  valid. 

-  The  planned  quantitative  validation  technique  based  on  comparing  the  missile  flyouts 
from  the  linked  runs  to  the  envelopes  resulting  from  Monte  Carlo  sampling  of  launch 
conditions  had  to  be  abandoned.  It  was  incapable  of  identifying  many  of  the  invalid 
lofting  missile  trajectories. 

..  The  validation  approach  was  modified  to  include  both  qualitative  and  quantitative 
validation  methods. 

—  Features  in  the  LPN-15  missile  flyout  were  used  for  qualitative  validation  of  the 
linked  results. 

—  Quantitative  validation  compared  the  missile  flyout  results  of  a  single  linked  run  to 
the  envelope  of  20  SIMLAB  standalone  runs  which  all  used  the  same  launch 
conditions  and  target  trajectory  as  the  linked  run. 

..  Applying  the  qualitative  method  to  the  Parametric  Study  Mission  V2  runs  showed  that 
the  missile  flyouts  were  valid  for  the  target  representation  in  the  SIMLAB  reference 
frame. 

-  Applying  the  quantitative  method  to  the  best  of  the  Parametric  Study  Mission  V2  runs 
showed  that  the  missile  flyouts  from  the  linked  runs  were  invalid  because  the  target 
representation  in  the  SIMLAB  reference  frame  was  in  error. 

—  Errors  in  initializing  the  target  and  missile  in  the  SIMLAB  reference  frame  were 
not  discovered  until  after  the  linked  runs  were  completed  and  have  since  been 
fixed. 


4-165 


...  Validity  of  the  missile  flyout  can  be  further  improved  by  more  accurate  SIMLAB 
integration  of  the  target  velocity  to  determine  target  position. 

-  Missile  TOF  gave  a  better  metric  for  quantifying  overall  missile  behavior  than  did  the 
miss  distance. 

Test  Objective  2:  Assess  utility  of  LSP  ADS  configuration  for  parametric  studies 

-  The  manual  method  for  replicating  a  given  profile  resulted  in  very  good  run-to-run 
reproducibility  of  the  launch  conditions. 

-  The  distribution  of  launch  conditions  results  was  significantly  tighter  than  the  shot  box 
tolerances  for  all  parameters. 

-  The  automatic  replay  method  was  unable  to  precisely  replicate  a  given  scenario. 

-  In  order  for  the  automatic  replay  method  to  be  effective,  the  manual  actions  required 
in  the  LSP  trials  need  to  be  replaced  with  automatic  procedures.  This  was  beyond  the 
scope  of  the  LSP,  but  future  implementations  of  this  concept  may  be  able  to  overcome 
this  limitation. 

Test  Objective  3:  Assess  effect  of  latency  on  validity  of  test  results 

-  Latency  Characteristics 

-  The  mean  latencies  of  the  entity  state  data  in  early  missions  were  large  with  large 
sample-to-sample  variations  during  a  run.  Shooter  entity  state  data  exhibited  very  high 
latency  “spikes”  with  latencies  up  to  ~1  sec. 

—  Most  of  the  latency  contribution  was  between  the  creating  simulation  and  the  MU 
where  PDUs  were  created  from  the  raw  simulation  data. 

—  Transmission  latencies  between  MUs  at  different  nodes  were  relatively  small  (~20 
msec). 

-  The  mean  latencies  of  all  entity  state  data  from  the  Parametric  Study  Mission  were 
relatively  small  and  consistent  run-to-run. 

—  The  latencies  between  any  pair  of  nodes  still  exhibited  relatively  large  sample-to- 
sample  variations  (relative  to  the  mean  value),  and  high  latency  “spikes”  were  still 
observed  (but  were  typically  only  up  to  ~  100  msec). 

—  The  latency  of  the  target  entity  state  data  between  the  MU  at  a  receiving  node  and 
the  receiving  simulation  was  about  twice  the  value  between  MUs. 

-  Latency  Effects 

—  Entity  Presentation  Errors 

—  Random  variations  in  latency  between  the  WSIC  and  the  SIMLAB  during  a  run 
resulted  in  an  uncertainty  in  the  target  location,  as  perceived  in  the  SIMLAB. 

—  Entity  velocity  data  exhibited  discontinuous  changes. 

--  Launch  Conditions  Differences  Between  Nodes 

—  Large  latencies  in  the  early  missions  (>200  msec)  resulted  in  significant 
disagreements  in  the  launch  conditions  perceived  by  different  simulations. 
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—  The  small  and  stable  latencies  in  the  Parametric  Study  Mission  (<200  msec) 
resulted  in  very  good  agreement  in  the  launch  conditions  perceived  by  different 
simulations.  Differences  were  typically  less  than  10%  of  the  shot  box  tolerances. 

—  The  Parametric  Study  Mission  results  show  that  the  shooter  and  target  simulations 
were  in  sufficient  agreement  to  allow  the  LSP  architecture  to  be  used  for  pre¬ 
launch,  closed-loop  interactions,  such  as  rehearsal  and  refinement  of  live 
engagement  scenarios. 

-  Terminal  Engagement  Conditions  Differences  Between  Nodes 

—  The  error  in  initializing  the  target  location  in  die  SIMLAB  simulation  and  the 
latitude  divergence  problem  prevented  a  direct  comparison  of  the  terminal  range  as 
determined  by  the  SIMLAB  simulation  with  the  terminal  range  as  determined  by 
entity  state  data  logged  at  other  locations. 

—  Results  from  entity  state  PDU  data  collected  at  different  nodes  imply  that  terminal 
engagement  results  for  a  closed-loop  interaction  between  the  missile  and  target 
would  be  invalid. 

Test  Subobjective  4-1:  Assess  capability  of  ADS  network  to  provide  bandwidth  and 

connectivity  required  for  LSP  tests 

-  Typical  bandwidth  utilization  was  low  (~4%)  on  the  T1  connecting  Point  Mu  gu  and  China 
Lake. 

-  There  was  no  loss  of  connectivity  during  the  LSP  missions. 

Test  Subobjective  4-2:  Assess  the  effects  of  ADS-induced  errors  on  LSP  test  results  validity 

-  There  were  no  missing  or  out-of-order  PDUs. 

-  The  errors  associated  with  transforming  the  entity  state  data  from  the  simulation  reference 
frame  to  the  PDU  reference  frame  and  back  again  were  small  and  acceptable  (1-2  ft.). 

-  Repeating  shooter  entity  state  PDUs  occurred.  However,  the  repeaters  stopped  5-10  sec 
before  launch,  so  that  no  errors  were  caused  in  the  launch  conditions. 

The  PDU  update  rates  for  the  various  entities  were  irregular.  This  aggravated  the  target 
latitude  divergence  problem. 

Test  Subobjective  4-3:  Assess  adequacy  of  standard  data  protocols  for  LSP  test 

-  The  DIS  PDUs  used  were  adequate  for  exchanging  information  between  the  simulations 
and  for  providing  data  for  the  test  control  displays. 

Test  Subobjective  4-4:  Assess  reliability,  availability,  and  maintainability  of  ADS  network 

-  The  average  availability  of  the  ADS  network  for  all  linked  testing  was  85.4%,  a  very 
acceptable  value. 

-  There  were  three  hardware  failures,  and  the  average  time  for  repairing  them  was  32  min. 

-  There  were  1 5  software  faults,  and  the  average  time  for  repairing  them  was  5  min. 
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Test  Subobjective  4-5:  Assess  capability  for  centralized  test  control  and  monitoring 

-  The  test  control  procedures  worked  quite  well. 

-  Monitors  provided  to  the  BMIC/TCAC  effectively  supported  test  control. 
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5.0  LSP  Lessons  Learned 

These  lessons  learned  incorporate  findings  from  the  NAWCWPNS  final  report  verbatim. 

5.1  Technical  Lessons  Learned 

5.1.1  Simulations 

-  Accurate  coordinate  transformations  are  necessary  .  Accurate  end-to-end  coordinate 
transformations  proved  difficult  at  the  beginning  of  the  LSP.  Coordinate  transformations 
must  be  verified  and  validated  at  each  site  and  then  reverified  and  revalidated  during  end- 
to-end  testing  as  early  as  possible  in  the  test  phase.  Personnel  who  are  subject  matter 
experts  in  coordinate  transformations  must  be  assigned  and  readily  available  during  this 
process. 

-  Quantitative  validation  has  limitations  .  The  JTF  intent  was  to  quantitatively  verify  missile 
simulation  performance  against  five  fire  data.  Given  the  facts  that  only  a  single  live  fly 
event  was  available  to  support  the  process  and  that  the  live  engagement  could  not  be 
perfectly  replicated,  it  became  necessary  to  modify  the  validation  approach.  The  modified 
approach  included  both  qualitative  and  quantitative  methods  (see  Section  4. 1.2.4)  and 
successfully  identified  invalid  results  (lofting  missile  trajectories  and  target  initialization 
errors). 

5.1.2  Interfaces 

-  Network  interfaces  need  improvement  .  Network  Interface  Units  (NIUs)  of  some  sort  are 
necessary  if  two  simulators  cannot  communicate  directly  in  a  common  language,  and  on  a 
common  timeline.  NIUs  can  be  a  major  source  of  both  error  and  processing  delays.  For 
the  LSP,  the  NIUs  were  difficult  to  troubleshoot  and  control.  Future  projects  should  use 
an  improved  DIS  NIU  or  an  “NIU  function”  (in  the  master  simulation  computer)  which 
provides  a  more  direct  user  control  of  the  content  of  the  data  and  network 
communications,  including  the  capability  to  force  network  communications  at  a  user- 
specified  frame  rate.  Such  improvements  could  simplify  the  overall  network/ADS 
configuration,  as  well  as  the  troubleshooting  and  resolution  of  various  network/ADS/DIS 
problems. 

5.1.3  Networks 

-  Common  ADS-related  hardware  and  software  is  needed  .  It  was  difficult  to  get  the  ADS 
network  to  behave  in  a  uniform  fashion  due  to  the  many  different  types  of  interface 
hardware,  communications  equipment  (  routers),  and  interface  software  versions.  In  all 
major  NAWCWPNS  simulation  operations  previous  to  the  LSP,  all  sites  had  exactly 


2  Joint  Advanced  Distributed  Simulation  Linked  Simulator  Phase  Final  Report.  NAWCWPNS  Report  Ser 
529100E/A-218,  16  January  1997. 
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duplicated  network  hardware  and  interface  software,  which  is  the  preferred  network 
architecture.  However,  program  schedule  and  cost  constraints  required  that  the  existing 
NAWCWPNS  NRNet  network  be  used  for  the  LSP  instead  of  building  a  network  from 
“scratch.”  Additionally,  lack  of  common  software  resulted  in  the  NIUs  having  numerous 
and  different  problems  related  to  conversions,  timing,  CPU  speed,  etc.  Whenever  possible 
for  future  ADS  test,  the  network  hardware  and  interface  software  should  be  exactly  the 
same  among  all  the  sites. 

Latency  variations  were  significant  .  Aggregate  latency  includes  the  transmission  delays 
and  the  processing  delays  on  each  leg  of  the  ADS  architecture.  Both  the  transmission- 
induced  and  processing  components  of  latency  exhibited  significant  random  variations 
which  cannot  be  compensated  for,  although  the  processing  delays  were  the  dominant 
source  of  latency.  Future  ADS  implementations  which  require  low  latencies  (e.g., 
interactive  missile  engagement  analyses)  should  focus  on  techniques  for  reducing  the 
latency  between  the  network  interface  and  the  receiving  simulations. 

.  Time  sources  must  be  synchronized  ,  IRIG/GPS  time  must  be  synchronized  off  of  the 
same  time  source  and  then  must  be  validated  at  each  test  site  prior  to  project  operations  to 
ensure  accurate,  synchronized  time  is  precisely  recorded  at  each  test  site.  It  took  a  lot  of 
effort  to  get  clock  synchronization  values  into  the  few  millisecond  region,  and  meaningful 
latency  measurements  were  impossible  without  this  degree  of  clock  synchronization. 

5.1.4  Instrumentation 

-  Special  test  equipment  is  needed  for  checkout  and  verification  .  Special  test  equipment 
(e.g.,  SNAP  loggers,  DIS  dataloggers,  etc.)  and  other  networking  tools  (e.g.,  Stealth 
Observer)  should  be  part  of  each  simulation  node’s  configuration  during  the  development, 
test,  checkout,  and  verification/validation  phases  in  all  subsequent  ADS  testing.  Special 
test  equipment  and  networking  tools  are  required  to  more  rapidly  isolate  and  determine 
the  specific  cause  of  network,  ADS/DIS,  etc.  problems.  Without  the  special  test 
equipment  and/or  tools,  trial  and  error  becomes  the  normal  troubleshooting  mode  which 
increases  the  resource  requirements  (time,  schedule,  cost,  etc.).  The  equipment  and  tools 
also  permit  off-line  testing  without  the  exclusive  scheduling  of  simulation  laboratories  and 
the  associated  costs.  Individual  test  sites  could  then  check  their  own  software  and 
hardware,  verify  PDUs  and  other  ADS/DIS  requirements,  and  verify  program-specific 
requirements  prior  to  the  more  costly  linked  tests.  The  special  test  equipment  and 
networking  tools  developed  and  used  by  the  JADS  JTF  for  the  LSP  proved  valuable  once 
the  equipment  and  tools  became  available  to  NAWCWPNS. 
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5.2  Infrastructure  Lessons  Learned 


5.2.1  Procedures 

5.2.1.1  Planning 

.  The  requirements  for  an  ADS  test  must  be  clearly  defined  early  in  the  test  planning  phase 

-  The  user  of  test  range  assets  (e.g.,  JADS)  must  clearly  define,  communicate,  and 
document  test  requirements  to  the  support  agency  (e.g.,  NAWCWPNS)  early  in  the 
test  planning  phase.  Also,  the  capability  of  the  support  agency  to  support  the  test 
must  be  clearly  stated  and  documented. 

-  OPSEC  requirements  for  a  distributed  weapon  system  test  must  be  determined  and 
coordinated  early  in  the  program,  especially  with  the  various  organizations  and  their 
different  procedures.  Issues  to  be  addressed  include:  the  need  for  an  OPSEC  Plan;  if 
an  OPSEC  Plan  is  required,  an  agreement  either  to  use  an  already  approved  OPSEC 
Plan  or  to  draft  a  new  OPSEC  Plan;  the  consistency  of  OPSEC  requirements  among 
the  various  organizations  and  programs;  OPSEC  requirements  in  test  control/conduct, 
including  the  use  of  “For  Official  Use  Only”  test  cards  and  step  calls  and/or  the  use  of 
secure  communications. 

—  Detailed  planning  and  coordination  will  be  required  to  ensure  a  common  understanding 
of  all  requirements,  procedures,  test  objectives,  etc.  since  individual  facilities  are  not 
generally  familiar  with  conducting  coordinated,  distributed  T&E  tests.  The 
communications,  test  control,  OPSEC,  and  security  aspects  are  different  between 
standalone  facility  tests  and  a  linked,  integrated,  weapon  system  test.  Minutes  and 
action  items  from  meetings  were  not  kept  during  the  LSP  and  should  be  a  part  of  any 
test/development  effort.  Maintaining  and  distributing  minutes  and  action  items  will 
provide  more  structured  and  cohesive  planning  and  coordination  efforts  which,  in  turn, 
should  assist  in  resolving  issues. 

.  SUT  experts  must  be  involved  from  the  beginning  .  Weapon  system(s)  analysis  experts 
must  be  planned  for  (and  budgeted  for)  to  analyze  the  weapon  system-related  data  from 
the  system  under  test  (SUT)  and  to  provide  the  analytical  results,  conclusions,  and 
recommendations.  There  should  be  more  than  one  expert,  and  they  must  be  involved  from 
the  beginning  of  the  project  to  establish  the  data  and  instrumentation  requirements, 
verily/validate  the  analytical  approach,  assist  in  the  development  of  test  matrices  and  test 
procedures,  and  provide  overall  weapon  system  expertise.  Options  include  the  support 
and/or  user  agencies  providing  the  SUT  analysis  experts.  Both  agencies  could  provide 
their  own  experts  who  would  independently  analyze  the  data  from  the  standpoint  of  the 
test  objectives.  The  independent  analyses  would  then  be  compared,  and  the  SUT  experts 
would  resolve  any  differences  in  their  conclusions. 

.  Test  communications  requirements  must  be  addressed  early  in  the  test  planning  phase 
~  This  is  necessary  to  ensure  effective  communications  during  the  test.  Remote  test 
control  using  two  non-secure  telephone  c  onference  bridges  (i.e.,  two  communications 
nets)  was  acceptable  .  However,  the  audio  level  between  connections  varied,  making 
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“loud  and  clear”  communications  among  all  the  sites  difficult  at  times  .  This  also 
caused  the  dynamic  range  of  the  recording  media  to  be  pushed  to  the  limit. 
Additionally,  the  limit  of  two  communications  nets  combined  with  the  high  ambient 
noise  at  the  laboratories  and  the  long  distances  between  the  telephone  outlet(s)  and  the 
required  location(s)  for  the  handset/headset  reduced  the  availability  of  communication 
options.  For  example,  the  F-14D  pilot  had  to  communicate  with  the  WSIC  operator 
via  voice  (i.e.,  shouting  due  to  the  distance  between  their  locations),  or  by  using  two 
headsets  (one  for  the  control  net  and  one  for  the  internal  WSIC  net),  or  by  talking  on 
the  control  net  which  would  have  added  extraneous  and  unnecessary  communications 
on  the  control  net. 

-  A  standard,  linked  test  should  have  multiple  (more  than  two)  communications  nets 
(e.g.,  control,  analyst,  network,  and  internal  laboratory)  with  easy,  selectable  access  to 
all  the  nets  from  multiple  locations  within  the  laboratory.  A  minimum  of  one  secure 
telephone  at  each  site  is  also  required.  More  complex,  linked  tests  may  require 
additional  non-secure  and/or  secure  communications  nets.  Speakers  with  plug-in 
push-to-talk  handsets  and/or  headsets  with  push-to-talk  switches  are  required  at 
strategic  locations  throughout  the  laboratory  to  allow  individuals  the  necessary 
mobility  and  communications  flexibility.  Speakers  are  also  necessary  when  VIP 
visitors  or  other  personnel  monitor  the  test  at  the  laboratory.  However,  laboratories 
should  not  be  required  to  duplicate  full  VIP  “accommodations”  that  should  already  be 
set  up  at  the  test  control  center.  Recommend  laboratories  research  various 
communication  options  by  reviewing  their  local  range  control  center  and  telemetry 
data  center  communication  setups. 

-  The  capability  for  secure  video  teleconferences  (  VTCs)  among  multiple  (more  than 
two)  sites  is  required  when  planning,  coordinating,  and/or  briefing  integrated  weapon 
system  tests  among  various  sites.  This  is  currently  a  problem  since  NAWCWPNS  only 
has  the  capability  for  a  secure  VTC  between  two  sites.  Additionally,  secure  telephone 
conference  bridges  are  also  required  when  a  test  must  be  conducted  using  secure 
communications  (e.g.,  a  special  project  test),  or  when  a  secure  VTC  is  not  available 
and  secure  briefs/debriefs  are  necessary. 

5.2.1.2  Development 

A  stepped  buildup  approach  should  be  used 

-  Systematic  checkout  of  the  standalone  simulators  is  needed  before  linking,  especially 
verification  of  simulation  laboratory  modifications  required  for  ADS  linking.  The 
modifications  should  be  performed  early  in  the  buildup  and  carefully  checked  and 
verified  before  linked  testing.  The  coordinate  transformations  needed  for  linked 
operation  are  a  particular  concern,  and  personnel  who  are  subject  matter  experts  in 
this  area  should  be  consulted  during  this  process. 

-  Direct  (non-DIS)  links  should  be  used  during  test  buildup.  This  focuses  early 
verification  checks  on  making  the  linked  architecture  work  without  the  additional 
interfaces  and  reference  frame  transformations  needed  for  DIS  implementation.  This 
also  provides  a  benchmark  for  the  DIS  implementation 
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~  Structured  testing  of  the  network  must  be  performed  prior  to ,  and  independent  of ,  the 
linked  testing  times  and  the  simulation  laboratories  to  validate  transmission/reception 
rates,  bandwidth  utilization,  latency,  data  transmission  and  reception,  etc.  prior  to 
commencing  project  test  periods.  A  more  in-depth  and  formal  validation  of  each 
segment  of  the  network  in  the  test  configuration  is  required  prior  to  project 
operations .  This  includes  a  b  etter  understanding  o  f  the  impact  of  each  network  device  , 
including  loggers ,  on  the  network ,  as  well  as  any  subnet  and  T1  issues,  during  test 
system  operation.  A  “test,  analyze,  fix,  test”  approach  in  combination  with  structured, 
independent  testing  of  the  network  during  the  LSP  would  have  been  beneficial.  In 
several  cases  during  the  LSP,  linked  testing  time  was  used  for  testing  the  network 
where  independent  network  testing  would  have  been  more  cost  effective. 

-  Linking  of  facilities  using  ADS  can  require  significant  facility  interface  hardware  and 
software  development  ADS  implementation  is  not  “plug  and  play.” 

5.2.1.3  Execution 

.  Local  (on-sitel  test  monitoring/control  should  be  used  prior  to  remote  test  monitoring/ _ 

control.  Numerous  questions  concerning  test  communications,  test  procedures,  and 
general  test  coordination  can  be  better  addressed,  and  more  quickly  resolved,  face-to-face. 
This  process  (local  and  then  remote  test  monitoring/control)  worked  well  during  the  LSP. 

.  Tight  control  of  the  aircrew  is  not  desirable  .  The  aircrew  should  be  allowed  to  perform  as 
aircrew  during  man-in-the-loop  testing.  Too  much  test  control  (e.g.,  “fire”  instead  of 
“cleared  to  fire”)  is  not  desirable  with  the  man-in-the-loop,  particularly  if  it  is  OT&E  or 
combined  DT&E/OT&E.  Testing  is  more  valuable  and  there  are  more  “lessons  learned” 
from  a  test  where  the  aircrew  are  given  the  critical  parameters  and  switchology  to  meet 
the  test  objective(s)  and  are  still  allowed  to  make  tactical  decisions,  fly  the  “aircraft,” 
operate  the  weapon  system,  etc.  A  tightly  controlled  test  is  appropriate  for  certain  testing 
such  as  computer  simulations  and  stand-alone  laboratory  tests. 

-  Additional  time  is  needed  before  die  beginning  and  after  the  end  of  each  testing  period — . 
Allocate  a  minimum  of  an  additional  two  hours  of  laboratory  time  at  the  end  of  each  test 
period  for  data  logging,  data  archiving,  data  transfer,  and  laboratory  reclassification  . 
Allocate  a  minimum  of  an  additional  one  hour  of  laboratory  set-up  time  prior  to  each  test 
period.  These  were  the  normal  pre-  and  post-test  laboratory  times  required  for  each  LSP 
formal  test  period.  The  pre-  and  post-test  requirements  should  be  included  in  the  number 
of  laboratory  hours  needed  for  each  test  period  and  incorporated  into  the  planned  costs. 

-  Briefings  are  needed  before  and  after  each  mission  .  Briefs  and  debriefs  should  be 
conducted  before  and  after  each  mission.  The  briefs  should  cover  such  items  as  the  test 
objectives,  telephone  numbers/frequencies  to  use  for  test  control,  test  configuration  of 
each  facility,  instrumentation  and  data  collection  requirements,  go/no  go  criteria, 
contingem^ack-up  plans,  test  conduct  including  a  detailed  review  of  test  cards, 
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communications  procedures,  OPSEC,  and  the  time  and  place  of  the  debrief.  A  briefing 
checklist  should  be  developed  and  used. 

5.2.1. 4  Evaluation 

.  Effective  data  management  is  needed  .  Linked  laboratories  can  generate  a  large  volume  of 
data  at  distributed  locations.  Without  careful  planning,  key  data  may  not  be  collected 
and/or  transmitted  to  the  analysis  center,  and  data  collected  at  the  sites  may  not  be  in  a 
useful  form  for  centralized  analysis.  A  comprehensive  data  management  plan  must  be 
developed  before  testing  which  clearly  identifies  (1)  the  data  to  be  collected  at  each  site, 
(2)  on-site  processing  of  the  data,  and  (3)  data  to  be  transmitted  to  the  analysis  center. 

-  Adequate  time  must  be  allotted  for  data  analysis  between  test  events  .  There  was  a 
tendency  to  underestimate  the  time  required  to  adequately  analyze  the  large  volume  of 
data  collected  in  the  test  events.  As  a  result,  some  problems  from  one  mission  were  not 
fully  diagnosed  and  fixed  before  the  next  mission.  In  fact,  some  problems  (e.g.,  target 
initialization  errors)  were  not  even  recognized  until  all  test  was  over.  Rehearsal  of  the 
analysis  procedures  should  be  used  to  better  estimate  the  time  required  for  adequate 
analysis  between  test  events. 

Tools  for  reading  and  analyzing  raw  simulation  data  should  be  ready  for  early  verification _ 

testing.  Verification  of  simulation  performance,  including  proper  inputs  from  other 
simulations,  requires  the  ability  to  read  and  process  the  raw  simulation  data.  Specialized 
tools  are  needed  for  this  purpose. 

5.2.2  Policy 

-  Future  ADS  T&E  projects  should  be  conducted  following  established  T&E  flight  test _ 

practices  and  procedures  .  The  LSP  was  more  typical  of  a  T&E  flight  test  effort  than  a 
standard  laboratory  test.  Specifically: 

-  Standard  Universal  Documentation  System  (UDS)  documents  such  as  requirements 
documentation  (e.g.,  Program  Introduction  (PI)  and/or  Operation  Requirements  (OR)) 
from  the  user  agency  and  response  documentation  (e.g.,  Statement  of  Capability 
(SOC)  and/or  Operations  Directive  (OD))  from  the  support  agency  should  be  used. 
This  would  establish  a  clear  set  of  requirements  at  the  beginning  of  the  program  from 
the  user  agency  and  a  clear  statement  of  the  support  agency’s  capabilities,  constraints, 
and  limitations  in  meeting  those  requirements.  A  Statement  of  Capability  was 
developed  by  NAWCWPNS  (the  support  agency)  for  the  LSP.  High-level  LSP 
functional  requirements  were  outlined  in  the  NAWCWPNS  Integration  Test  Plan. 

—  Test  monitoring/control  and  test  procedures/conduct  should  be  run  similar  to  a  flight 
test.  Detailed  test  cards  should  be  drafted,  reviewed,  distributed,  briefed,  and  used  for 
each  mission.  Back-up  test  cards  should  also  be  considered  and  briefed  for 
contingency  purposes.  Briefs  and  debriefs  should  be  conducted  before  and  after  each 
mission.  The  briefs  should  cover  such  items  as  the  test  objectives;  telephone 
numbers/frequencies  to  use  for  test  control,  etc.;  test  configuration  of  each  laboratory; 
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instrumentation  and  data  collection  requirements;  go/no  go  criteria;  contingency/  back¬ 
up  plans;  test  conduct  including  a  detailed  review  of  the  test  cards;  communications 
procedures;  OPSEC;  and  the  time  and  place  of  the  debrief.  A  briefing  checklist  should 
be  developed  and  used.  The  LSP  used  one  basic  profile  which  permitted  simplified 
test  cards  and  test  procedures.  These  simplified  cards  and  procedures,  including  the 
use  of  a  few  “step”  calls,  were  satisfactory  for  the  LSP. 

Configuration  control  is  essential  .  Configuration  control  of  the  network  and  the  ADS/DIS 
system,  including  its  hardware,  software,  and  its  simulator  interfaces,  is  necessary  starting 
at  the  beginning  of  the  program.  This  includes  a  “scientific”  approach  to  network 
management  and  troubleshooting.  Either  a  single  person  or  a  network  committee  should 
be  in  charge  of  the  configuration  control/network  management.  The  level  of  control  will 
vary  with  the  phase  of  the  project.  Since  problems  are  part  of  the  process,  the  network 
configuration  cannot  be  “frozen”  until  there  is  an  agreed  upon  “baseline.”  However,  the 
configuration  control  process/procedures,  individual/committee  in  charge,  etc.  must  be 
established  at  the  beginning  of  the  program  and  followed  until  the  end  of  the  program. 

5.2.3  Costs 

-  The  LSP  development  effort  should  have  been  negotiated  as  a  “cost”  type  contract  with 
NAWCWPNS .  The  LSP  program  should  not  have  been  negotiated  as  a  “firm  fixed  price” 
contract.  Just  like  a  user  “buys”  hours  of  range  time,  number  of  target  presentations, 
specific  instrumentation/data  packages,  etc.,  on  a  flight  test  program,  so  should  a  user 
“buy”  hours  of  laboratory  time,  etc.  for  future  ADS  exercises,  particularly  those  that  test 
an  integrated  missile  weapon/launch  aircraft  system. 

—  A  “firm  fixed  price”  contract  assumes  all  the  requirements  are  completely  and 
accurately  specified  at  the  beginning  of  the  program.  Since  project  requirements 
historically  change  over  the  course  of  a  program,  and  assumptions  have  to  be  made 
concerning  the  specific  details  of  the  requirements  during  the  initial  planning/costing 
stage,  it  is  difficult  to  accurately  forecast  total  costs  and  hold  to  a  “firm  fixed  price.” 
As  the  requirements  change,  early  assumptions  are  proved  incorrect,  and/or  technical 
knowledge  is  gained  on  the  program,  as  occurred  during  the  LSP,  and  a  “cost”  type 
contract  provides  both  the  user  and  the  support  agencies  with  more  flexibility. 
Information  is  readily  available  as  to  the  costs  which  may  be  incurred  if  the  user 
agency  has  to  vary  the  number  of  missions,  hours,  etc.  due  to  unforeseen 
circumstances  and/or  problems. 

-  A  “cost”  type  contract  also  reduces  the  potential  for  budget/funding  disagreements 
during  the  course  of  the  program.  The  user  agency  can  more  easily  evaluate  the 
options  as  things  change  during  the  project  operations  phase  and  then  make  the 
necessary  trade-  offs  since  the  required  cost  information  (e.g.,  the  cost  per  hour  for 
each  laboratory)  has  already  been  provided. 

—  A  canceled/ incompleted  operations  allowance  should  be  budgeted  for  a  linked 
laboratory  test  just  as  it  is  for  a  flight  test.  This  allowance  “plans”  for  missions  that  are 
canceled  late,  therefore  incurring  full  or  partial  costs.  It  also  “plans”  for  missions 
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where  the  test  objectives  are  only  partially  accomplished  and  another  mission  is 
necessary  to  complete  the  test  objectives. 


5.2.4  Personnel 

PIS  training  early  in  the  program  was  beneficial  .  The  DIS  class  attended  by  the  three 
primary  NAWCWPNS  engineers  clarified  DIS  and  unified  their  vision  of  what  the  DIS 
portion  of  the  test  would  entail.  ADS/DIS  and  other  specialized  training  for  the  primary 
and  supporting  engineers  should  be  planned,  budgeted,  and  conducted  very  early  in  ADS 
program. 
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6.0  Conclusions/Recommendations 


6.1  Utility 

6.1.1  Utility  Conclusions 

-  The  LSP  ADS  configuration  has  utility  for  missile  weapon/launch  aircraft  system  T&E. 

-  The  configuration  successfully  ran  inte  grated  scenarios/profiles  among  linked 
laboratories. 

-  This  configuration  can  be  used  for  discrepancy/deficiency  resolution,  especially  when 
there  are  interface  issues/problems  between/among  weapon  systems  (e.g.,  the  aircraft 
radar,  mission  computer,  stores  management  system,  and  the  missile).  This  includes 
troubleshooting  problems  which  prove  to  be  difficult  to  replicate,  particularly  those 
that  appear  in  flight  tests  but  are  not  readily  duplicated  in  stand-alone  laboratory 
testing. 

—  Linked  laboratorie  s  permit  the  HWIL  missile  to  respond  to  actual  pre-  and  post-launch 
weapon  system  inputs,  instead  of  relying  on  stand-alone  “canned”  inputs,  in  a  more 
operationally  realistic  environment 

-  The  LSP  ADS  configuration  has  utility  for  parametric  studies  involving  a  one-on-one  air- 

to-air  missile  engagement. 

—  The  key  characteristic  of  a  parametric  study  is  the  ability  to  repeat  a  given  scenario 
with  either  no  changes  or  with  a  single  parameter  varying. 

—  The  manual  method  for  replicating  a  given  profile  resul  ted  in  veiy  good  run-to-run 
reproducibility  of  the  launch  conditions. 

-  The  automatic  replay  method,  as  implemented  in  the  LSP,  was  unable  to  precisely 
replicate  a  given  scenario  and  had  no  advantage  over  the  manual  method. 

-  The  LSP  ADS  configuration  has  utility  for  rehearsal  and  refinement  of  live  engagement 

scenarios. 

-  Pilot  training  and  rehearsals  of  five  missile  firings  requiring  difficult  and/or  precise 
launch  conditions  could  be  accomplished  using  this  configuration.  The  ADS  link 
could  be  used  by  the  pilot  to  practice  (pre-fly)  the  mission  in  a  controlled  laboratory 
environment  before  using  aircraft  and  range  assets.  ADS  could  assist  in  doing  the 
flight  test  right  the  first  time  which  translates  into  reduced  aircraft  flight  hours,  range 
time,  etc. 

-  The  LSP  ADS  configuration  does  not  have  utility  for  OT&E,  tactics  development,  and 

“free  play”  scenarios. 

-  The  aircraft  weapon  system  laboratories  did  not  have  totally  realistic  flying/handling 
qualities  when  compared  to  the  actual  aircraft  or  sophisticated  visual  presentation 
systems;  they  were  primarily  designed  for  stand-alone  weapon  system  testing. 

--  The  missile  HWIL  laboratory  could  not  handle  large  angles  off  and/or  large  angle  rates 
to  accommodate  the  above  scenarios. 
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-  The  LSP  ADS  configuration  does  not  have  utility  for  terminal  engagement  studies 
involving  closed-loop  interactions  between  the  missile  and  the  target. 

—  Target  position  errors  and  latencies  were  such  that  the  nodes  disagreed  on  the  final 
range  between  the  missile  and  the  target  by  more  than  the  missile  lethal  radius.  Hence, 
the  nodes  could  disagree  on  whether  or  not  the  target  had  been  “killed.” 

6.1.2  Utility  Recommendations 

-  Develop  a  flight  test  data  playback  capability  for  the  laboratories  which  can  also  be 
precisely  controlled/coordinated  between/among  laboratories. 

—  Aircraft  weapon  system  and  missile  system  laboratories  should  have  the  capability  to 
replay  flight  test  data  (i.e.,  weapon  system,  missile,  sensor,  mission  computer,  TSPI, 
etc.)  as  a  method  to  “drive”  the  flight  profile  of  the  laboratory  and/or  its  high  fidelity 
simulation. 

—  There  should  also  be  options  for  user-  selectable  full  or  partial/subset  replay  of  the 
data.  For  example,  the  user  may  only  be  interested  in  replaying  the  SMS  data  on  the 
avionics  bus  without  having  to  replay  all  the  data  from  all  the  buses. 

—  Additionally,  automated  control  (e.g.,  using  the  simulation  management  PDUs)  is 
required  when  attempting  to  exactly  replicate/replay  a  specific  profile,  set  of 
parameters,  or  series  of  events. 

-  The  LSP  utility  can  be  extended  to  OT&E,  tactics  development,  and  “free  play”  scenarios 
by  linking  more  realistic  aircraft  and  missile  HWIL  laboratories  which  are  able  to 
accommodate  these  scenarios. 

6.2  Technical 

6.2.1  Technical  Conclusions 

-  The  LSP  test  objectives  were  successfully  accomplished. 

—  The  V&V  method  implemented  identified  errors  in  interfacing  the  target  and  shooter 
entity  state  data  to  the  missile  HWIL  simulation. 

—  The  LSP  ADS  configuration  was  shown  to  have  utility  for  p  arametric  studies  involving 
an  air-to-air  missile  engagement  with  IRCM. 

—  The  latency  of  the  LSP  ADS  configuration  was  evaluated  and  determined  to  be 
sufficiently  small  such  that  the  shooter  and  target  entities  could  agree  on  launch 
conditions  to  within  10%  of  the  shot  box. 

—  The  LSP  ADS  configuration  was  assessed  as  being  able  to  support  AIM-9  testing, 
with  the  exception  of  closely-coupled  terminal  engagements  between  the  missile  and 
the  target. 

-  The  sample-to-sample  variations  in  latency  during  a  run  represent  a  fundamental  limitation 
to  low-latency  applications.  The  standard  deviations  of  the  latencies  were  typically 
significant  fractions  of  the  mean  values  (~35%-60%).  These  random  latency  variations 
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cannot  be  eliminated  by  deterministic  corrections  and  resulted  in  an  uncertainty  in  the 
target  position  as  observed  at  the  SIMLAB  on  the  order  of  30  ft. 

-  Future  ADS  applications  for  evaluation  of  closely-coupled  terminal  engagements  between 
the  missile  and  target  will  require  special  design  to  achieve  sufficient  simulation 
synchronization  and  low  latencies.  The  simulation  computers  must  have  their  frame  rates 
synchronized  and  latencies  must  be  reduced  to  less  than  the  frame  time.  This  degree  of 
synchronization  will  require  modifications  to  the  simulation  computers  themselves,  or  else 
ADS  must  be  designed  into  the  simulation  computers  from  the  start. 

6.2.2  Technical  Recommendations 

Future  ADS  applications  for  missile  T&E  should  use  an  improved  NIU  or  an  “NIU 
function”  (in  the  master  simulation  computer).  The  objective  would  be  to  provide  a  more 
direct  user  control  of  the  content  of  the  data  and  network  communications,  including  the 
capability  to  force  network  communications  at  a  user-specified  frame  rate.  Such 
improvements  could  simplify  the  overall  network/ADS  configuration,  as  well  as  the 
troubleshooting  and  resolution  of  various  network/ADS/DIS  problems. 

6.3  Infrastructure 

6.3.1  Infrastructure  Conclusions 

-  While  this  was  a  relatively  simple  architecture,  the  set-up  and  checkout  activities 
consumed  significantly  more  time  than  planned.  A  dwindling  number  of  people  have  an 
expectation  that  creating  a  linked  T&E  architecture  is  a  “plug-and-play”  exercise.  This 
test  phase  clearly  showed  it  is  not. 

A  sequential,  build-up  approach  to  verifying  network  performance  is  necessary.  Initially,  a 
lot  of  attention  needs  to  be  paid  to  the  simulators  and  simulations  in  a  standalone  mode  so 
that  network-driven  modifications  can  be  checked  out.  Only  when  each  node  has  been 
judged  “healthy”  does  it  make  sense  to  embark  upon  the  assessment  of  the  integrated 
architecture. 

A  full-up  linked  architecture  is  necessary  to  validate  “fixes.”  Many  fixes  cannot  be 
adequately  assessed  unless  the  entire  network  is  used.  Test  planners  should  incorporate 
these  linked  tesing  periods  in  their  schedules  and  budgets  —  they  are  not  any  cheaper  than 
rehearsals. 

-  Test  planners  engaged  in  using  this  kind  of  architecture  should  probably  plan  for  two 
attempts  on  every  required  mission.  (Bear  in  mind  this  test  only  had  three  participating 
nodes,  and  a  control  node  —  as  the  number  of  nodes  goes  up,  the  planner’s  expectations 
should  go  down.)  Additional  time  prior  to  the  start  of  test,  and  at  the  end  of  each  test 
period  is  necessary.  The  former  to  fine  tune  the  network,  and  eliminate  start-up  glitches, 
and  the  latter  to  accommodate  data  logging,  archiving,  and  transfer. 
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-  Test  control  worked  well,  whether  implemented  from  the  BMIC  or  the  TCAC.  In  a 
distributed  test  architecture,  the  control  mechanism  must  support  a  sensible  blend  of 
centralization,  test  direction,  level  of  control,  and  decentralization,  test  execution,  node¬ 
level  controls.  The  emergencies  usually  occur  at  the  node  level;  adjustments  after 
emergencies  are  usually  best  managed  at  the  test  direction  level. 

-  Configuration  control,  and  the  associated  documentation,  is  essential  to  a  successfiil  test 
program.  An  adequate  management  structure,  with  the  requisite  authority,  must  be  put  in 
place  prior  to  the  start  of  testing. 

-  The  SUT  experts  must  be  invo  lved  very  early  in  the  network  architecture  design  process. 
The  elements  of  the  network  interact,  sometimes  in  subtle  ways,  with  the  systems  and 
subsystems  of  the  SUT. 

6.3.2  Infrastructure  Recommendations 

-  Future  ADS  T&E  projects  should  be  conducted  following  established  T&E  flight  test 
practices  and  procedures.  This  is  especially  important  when  linking  laboratories  which  are 
not  usually  involved  in  T&E. 

-  Future  ADS  development  efforts  should  be  negotiated  with  the  range/test  facilities  as 
“cost”  type  contracts,  rather  than  “firm  fixed  price”  contracts. 
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APPENDIX  A 


CLASSIFIED  SUPPLEMENT 
for  the 

SYSTEM  INTEGRATION  TEST 
LINKED  SIMULATORS  PHASE 


(AVAILABLE  UNDER  SEPARATE  COVER  FROM  JADS  JTF) 


APPENDIX  B 


SECURITY 
for  the 

SYSTEM  INTEGRATION  TEST 


LINKED  SIMULATORS  PHASE 


MEMORANDUM  OF  AGREEMENT 
AMONG  THE  USERS  OF  THE 
NAWCWPNS  REALTIME  NETWORK 
(NRNet) 


1.0  INTRODUCTION: 

The  Naval  Air  Warfare  Center  Weapons  Division  (NAWCWPNS)  Realtime 
Network  (NRNet)  is  a  secured  network  capable  of  transporting  up  to  and  including 
SECRET  information.  NRNet  contains  point-to-point  T-Carrier  communications  links 
covered  by  National  Security  Agency  approved  cryptographic  equipment  (KG- 194,  KIV- 
7,  etc.).  NRNet  is  a  high  capacity  network  supporting  a  full  spectrum  of  warfighting  test 
and  evaluation,  training,  and  simulation  interoperability  activities.  This  Memorandum  of 
Agreement  (MO A)  contains  the  requirements  for  interconnection  and  secure  operation  of 
user  systems  connected  to  the  NRNet. 

2.0  RULES  FOR  CONNECTION  AND  SECURE  OPERATIO N  OF  THE  NRNet: 

Each  NRNet  User  System  Designated  Approving  Authority  (DAA)  must  agree  to 
this  MOA  and  take  responsibility  for  ensuring  proper  secure  operation  by  signing  this 
agreement.  All  NRNet  User  Systems  must  meet  the  rules  for  connection  and  ensure 
secure  operation  on  the  NRNet.  The  NRNet  DAA  will  adjudicate  accreditation 
differences  between  each  NRNet  User  System  DAA.  Because  of  the  multiservice/agency 
nature  of  the  NRNet,  each  User  System  is  accredited  for  the  dedicated  Mode  of  Operation 
at  the  SECRET  security  level  under  the  provisions  of  DoD  Directive  5200.28,  Security 
Requirements  for  Automated  Information  Systems  (AIS). 

The  boundary  of  the  NRNet  is  defined  to  be  the  unencrypted/clear  Ethernet 
interface  at  each  NRNet  Router.  The  NRNet  DAA  has  security  responsibility  from  that 
interface  through  the  NRNet  while  the  User  System  DAA  has  security  responsibility  from 
that  interface  back  into  the  User’s  System.  The  User  System  DAA  has  responsibility  to 
ensure  that  all  AIS  connected  in  any  way  to  the  NRNet  meet  the  rules  for  connection  and 
secure  operation  of  the  NRNet. 

As  a  minimum  AIS  Security  requirement,  all  AIS  must  at  least  meet  DoD  5200.28 
requirements  for  the  dedicated  Mode  of  Operation  SECRET  security  level.  The 
accreditation  letter/document  for  the  USER  System  will  be  provided  by  the  User  System 
DAA  and  will  become  part  of  this  MOA. 

If  the  User  System  covered  in  this  MOA  provides  connection  to  an  AIS  not 
managed  by  the  User  System  DAA,  the  DAA  for  the  User  System  will  establish  an  MOA 
with  connecting  AIS  DAA  in  accordance  with  the  requirements  of  DoD  Directive 
5200.28,  that  ensures  continued  compliance  with  the  rules  for  connection  and  secure 
operation  of  the  NRNet 


B-l 


User  Systems  must  meet  all  installation,  physical  protection,  accounting, 
procedural  and  access  control  protection  mechanisms  required  for  SECRET  operation  of 
the  NRNet  and  the  accredited  AIS  at  their  site. 

Every  User  System  agrees  to  operate  the  NRNet  in  accordance  with  the  doctrine 
provided  by  the  National  Security  Agency  (NS A)  andNRNet  Management. 

User  Systems  must  have  Communication  Security  (COMSEC)  Material  System 
(CMS)  accounts  and  must  use  NRNet  Administrator  approved  NSA  provided  keying 
material  for  NRNet  managed  cryptographic  devices. 

User  Systems  may  be  untrusted  systems  operating  in  the  dedicated  mode  at  the 
SECRET  (NOFORN)  security  level.  This  is  expected  to  be  the  normal  Mode  of 
Operation  for  every  AIS  connected  to  the  NRNet.  In  this  mode  of  operation,  all  users 
must  have  the  appropriate  clearance  and  need-to-know  for  all  data  handled  by  their  AIS. 
Additionally,  the  User  System  DAA  acknowledges  that  all  users  of  the  NRNet  may  have 
access  to  the  data  contained  within  their  accredited  AIS. 

User  Systems  may  be  untrusted  systems,  operating  in  the  system  high  SECRET 
security  level.  User  Systems  accredited  to  operate  in  this  mode,  which  connect  to  the 
NRNet,  acknowledge  that  the  NRNet  provides  no  protection  within  the  NRNet  or  other 
user  systems  beyond  those  of  DoD  Directive  5200.28  accredited  dedicated  level  mode  of 
operation.  User  Systems  accredited  to  operate  in  the  system  high  SECRET  security  level 
mode  accept  responsibility  for  the  possibility  of  increased  risk  to  their  AIS  because  of  the 
interconnection  with  systems  accredited  to  operate  in  the  dedicated  Mode  of  Operation. 

User  Systems  may  connect  with  a  single  security  level  of  SECRET,  labeled  port  on 
a  trusted  AIS  accredited  to  operate  in  the  multilevel  mode  with  the  risk  range  identified  in 
the  DoD  Directive  5200.28.  User  Systems  accredited  to  operate  in  this  mode,  which 
connect  to  the  NRNet,  acknowledge  that  the  NRNet  provides  no  protection  within  it’s 
network  or  other  User  Systems  beyond  those  of  DoD  Directive  5200.28  dedicated  Mode 
of  Operation.  User  Systems  accredited  to  operate  in  the  multilevel  mode  accept 
responsibility  for  the  possibility  of  increased  risk  to  the  AIS  because  of  the  interconnection 
with  systems  accredited  to  operate  in  the  dedicated  mode. 

User  Systems  that  employ  trusted  operating  systems  must  operate  the  systems  in 
accordance  with  the  systems’  Security  Features  Users  Guide  and  the  Trusted  Facility 
Manual. 

User  Systems  must  treat  all  data  or  information  via  the  NRNet  Ports  as  classified 
at  the  SECRET  level  until  the  data  or  information  has  been  manually  reviewed  and 
downgraded.  Permanent  storage  media  for  data  or  information  will  be  labeled  and 
controlled  at  the  SECRET  level. 
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Each  User  System  must  identify  its  facility  accreditation  authority  and  provide 

evidence  of  site  accreditation  at  the  SECRET  level  before  connection  to  theNRNet. 

3.0  USER  SYSTEMS  SPECIFIC  INFORMATION: 

3 . 1  Name  of  POC  and  AIS  Component  Connected  to  the  NRNet: 

3.1.1  Naval  Air  Warfare  Center  Weapons  Division 
Sea  Range  -  Communications  Building 
Building  531,  Point  Mugu,  CA  93042 

AIS  System  Security  Officer:  Joel  Bossoletti 

Code  524300E 
DSN  351-0902 
Com  (805)  989-0902 

3.1.2  Naval  Air  Warfare  Center  Weapons  Division 

F-14D  Weapons  Support  Integration  Capability  (WSIC) 

Building  761,  Point  Mugu,  CA  93042 

AIS  System  Security  Officer:  Brian  Krinsley 

Code  41L200E 
DSN  351-9007 
Com  (805)  989-9007 


3.1.3  Naval  Air  Warfare  Center  Weapons  Division 

Land  Range  -  Range  Control  Center 
Building  31455,  China  Lake,  CA  93555-6001 
AIS  System  Security  Officer:  Bob  Eisenhauer 

Code  524500D 
DSN  437-6717 
Com  (619)  939-6717 

3.1.4  Naval  Air  Warfare  Center  Weapons  Division 
F/A-18  Weapons  Systems  Support  Facility  (WSSF) 
Building  20279,  China  Lake,  CA  93555-6001 

AIS  System  Security  Officer:  Wray  Jacobs 

Code  413 100D 
DSN  437-5106 
Com  (619)939-5106 
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3.1.5 


Naval  Air  Warfare  Center  Weapons  Division 
Simulation  Laboratory  (SIMLAB) 

Building  00005,  China  Lake,  CA  93555-6001 
AIS  System  Security  Officer:  Ed  Slack 

Code 471700D 
DSN  437-1457 
Com  (619)939-1457 

3.1.6  Naval  Air  Warfare  Center  Weapons  Division 
Weapons  Tactics  (WEPT AC) 

Building  03264,  China  Lake,  CA  93555-6001 
AIS  System  Security  Officer:  Mark  Hilde 

Code  4J5100D 
DSN  469-1929 
Com  (619)  927-1929 

3.1.7  US  Air  Force 
TCAC 

1951  2nd  Street  SE 
Kirtland  AFB,  NM  87117-5617 

AIS  System  Security  Officer:  Lt  Col  Homer  L.  Jeffers,  USAF 

Code  JADS  JTF 
DSN  246-0528 
Com  (505)  846-0528 


3 .2  Highest  Classification  of  Data  or  Information  processed  on  theAIS : 

US  SECRET 

3.3  Approval  Date: 

15  April  1996 

3.4  Accredited  Operation  Mode: 

SYSTEM  HIGH  SECRET  (NOFORN) 

4.0  USER  SYSTEM  CERTIFICATION: 

As  the  DAA  for  my  NRNet  System,  I  hereby  certify  that  the  area  that  houses  my 
NRNet  Nodes(s)  has  a  facility  accreditation  approval  for  the  operation  at  SYSTEM 
HIGH  SECRET,  while  connected  to  NRNet.  I  further  certify  that  the  proper  policies  and 
procedures  are  in  place  to  ensure  that  the  AIS/NRNet  equipment  at  my  site  will  be 
operated  in  a  manner  that  meets  the  NSA  Doctrine  for  the  encryption  equipment  used  and 
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the  DoD  Directive  5200.28  requirements  for  SECRET  (  NOFOEN)  security  level  in  the 
system  high  mode. 

I  concur  with  this  Memorandum  of  Agreement 
Date:  _ 

4.1  Name:  RADM  Dana  B.  McKinney,  USN 

Title:  Designated  Approving  Authority 

Naval  Air  Warfare  Center  Weapons  Division 

Signature: _ _ 

4.2  Name:  Lt  Col  Homer  L.  Jeffers,  US AF 

Title:  Designated  Approving  Authority 
JADS  JTF 

Signature: _ _ _ 
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APPENDIX  C 

INTEGRATED  DATA  REQUIREMENTS  LIST 

for  the 

SYSTEM  INTEGRATION  TEST 
LINKED  SIMULATORS  PHASE 


LIST  OF  TABLES 


Page 

Table  C-l.  General  Test  Parameters  C-l 

Table  C-2.  Parameters  Recorded  at  F/A-l 8  WSSF  C-2 

Table  C-3.  Parameters  Recorded  at  F-14D  WSIC  C-4 

Table  C-4.  Parameters  Recorded  at  AIM-9M  SIMLAB  C-5 

Table  C-5.  Parameters  Recorded  at  TCAC  C-9 

ANNEX  1  -  PDU  DESCRIPTIONS  Cl-1 


Table  C-l.  General  Test  Parameters 


Data  Requirement 

Description 

Units 

EEEftSsSSSHl 

Test  Name 

Linked  Simulators  Phase 

N/A 

Once 

Mission  Number 

Mission  identification 
number  from  TAP 

N/A 

Each 

mission 

Profile  Number 

Profile  identification 
number  from  TAP 

N/A 

Each  profile 

Trial  Number 

Trial  identification  number 
from  test  sequence 

N/A 

Each  trial 

Trial  Date 

Date  of  trial 

DD-MM-YY 

Each  trial 

Trial  Start  Time 

Time  at  TCAC  when  start 
trial  command  issued 

HH:MM:SS.S 

SS 

Each  trial 

Trial  Hold  Time 

Time  at  TCAC  when  hold 
trial  command  issued 

HH:MM:SS.S 

SS 

Each  hold,  if 
applicable 

Trial  Restart  Time 

Time  at  TCAC  when 
restart  trial  command 
issued 

HH:MM:SS.S 

SS 

Each  restart, 
if  applicable 

Trial  Stop  Time 

Time  at  TCAC  when  stop 
trial  command  issued 

HH:MM:SS.S 

SS 

Each  trial 

Trial  Problems 

Any  problem  encountered 
in  executing  trial.  Reason 
for  trial  hold. 

N/A 

Each 

trial/hold,  if 
applicable 
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Table  C-2.  Parameters  Recorded  at  F/A-18  WSSF 


Data  Requirement 

Description 

Units 

Shooter  entity  state  data 

WSSF  sim  computer  output 
for  shooter 

Each  trial 

Time 

UTC  time  in  IRIG-B  format 

Latitude 

Radians 

Longitude 

Radians 

Altitude 

Feet 

Heading,  True 

Radians 

Pitch  &  Roll 

Radians 

Pitch,  Roll,  &  Yaw  Rates 

Radians/Sec 

Velocities  (NED) 

Feet/Sec 

Accelerations  (NED) 

Entity  ID 

Numeric 

Target  entity  state  data 

Logged  by  WSSF  sim  from 
WSIC 

Each  trial 

Time 

UTC  time  in  IRIG-B  format 

Latitude 

Radians 

Longitude 

Radians 

Altitude 

Feet 

Heading,  True 

Radians 

Pitch  &  Roll 

Radians 

Pitch,  Roll,  &  Yaw  Rates 

Radians/Sec 

Velocities  (NED) 

Feet/Sec 

Accelerations  (NED) 

IsmrcflSflH 

Entity  ID 

Numeric 

Missile  entity  state  data 

Logged  by  WSSF  sim  from 
SIMLAB 

Each  trial 

Time 

UTC  time  in  IRIG-B  format 

Latitude 

Radians 

Longitude 

Radians 

Emm 

Altitude 

Feet 

Heading,  True 

Radians 

Pitch  &  Roll 

Radians 

Pitch,  Roll,  &  Yaw  Rates 

Radians/Sec 

Velocities  (NED) 

Feet/Sec 

|  Entity  ID 

Numeric 

Table  C-2.  Parameters  Recorded  at  F/A-18  WSSF  (Concluded) 


Data  Requirement 

Description 

Units 

PDU  Data 

Logged  on  WSSF  STRICOM 
Logger 

Shooter  entity  state  PDU 

Generated  by  WSSF,  see 

Table  Cl-1 

Each  trial, 
when  sent 

Target  entity  state  PDU 

Received  at  WSSF  from 

WSIC,  see  Table  Cl-2 

Each  trial, 
upon  receipt 

Missile  entity  state  PDU 

Received  at  WSSF  from 
SIMLAB,  see  Table  Cl-3 

Each  trial, 
upon  receipt 

Fire  (flare)  PDU 

Received  at  WSSF  from 

WSIC,  see  Table  Cl-4 

Each  trial, 
once 

Fire  (missile)  PDU 

Received  at  WSSF  from 
SIMLAB,  see  Table  Cl-5 

Each  trial, 
once 

Detonation  PDU 

Received  at  WSSF  from 
SIMLAB,  see  Table  Cl-6 

Each  trial, 
once 

once 


Table  C-3.  Parameters  Recorded  at  F-14D  WSIC 


Data  Requirement 

Description 

Units 

Target  entity  state  data 

WSIC  sim  computer  output 
for  target 

Each  trial 

Time 

UTC  time  in  IRIG-B  format 

Latitude 

Degrees, 

Minutes 

Every  25  ms 

Longitude 

Degrees, 

Minutes 

Every  25  ms 

Altitude 

Feet 

Every  25  ms 

Heading,  True 

WSIC  only  data 

Pitch,  Roll 

Yaw 

From  NIU  recording  only 

Pitch,  Roll,  &  Yaw  Rates 

Velocities  (NED) 

Feet/Sec 

Accelerations  (NED) 

Entity  ID 

From  NIU  recording  only 

Numeric 

PDU  Data 

Logged  on  WSIC  STRICOM 
Logger 

Target  entity  state  PDU 

Generated  by  WSIC,  see  Table 
Cl-2 

Each  trial, 
when  sent 

Fire  (flare)  PDU 

Generated  by  WSIC,  see  Table 
Cl-4 

Each  trial, 
once 

Shooter  entity  state  PDU 

Received  at  WSIC  from 

WSSF,  see  Table  Cl-1 

Each  trial, 
upon  receipt 

Missile  entity  state  PDU 

Received  at  WSIC  from 
SIMLAB,  see  Table  Cl -3 

Each  trial, 
upon  receipt 

Fire  (missile)  PDU 

Received  at  WSIC  from 
SIMLAB,  see  Table  Cl -5 

Each  trial, 
once 

Detonation  PDU 

Received  at  WSIC  from 
SIMLAB,  see  Table  Cl-6 

Each  trial, 
once 
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Table  C-4.  Parameters  Recorded  at  AIM-9M  SIMLAB 


Data  Requirement 


Simulation  and  Facility 
Data 


S  Y  STEM_TIME 


IRIG  TIME 


DELTA_RL 


DELTA  UD 


PHIDOT_AZ 


PHIDOTEL 


CSA_RL 


CSA  UD 


COMMAND_RL_HDW 


COMMAND_UD_HDW 


LAMBDA  A Z 


LAMBDA  EL 


PHIW  _ 


PHI 


THETA  CARCO 


THETA  CT  FB 


THETA.ERR 


PHI  CT 


PHI  CT  FB 


PSI  CARCO 


PSI  CT  FB 


PSI_ERR 


MIRROR_AZ_CMD 


MIR  AZ  FB 


mirror_el_cmd 


MIR  EL_FB 


TRANS_CMD 


MIR_TRANS_FB 


FLARE_FLAG 


Description 


Internal  SIMLAB  simulation 
parameters 


Simulation  Time  from  Start  of 
Missile  Flyout 


IRIG-B  Time 


Azimuth  Seeker  Precession  Rate 


Elevation  Seeker  Precession  Rate 


Servo  Amp  Output  (R/L 


Servo  Amp  Output  (U/D 


Seeker  Hardware  Unsealed 
Guidance  Command  (R/L) 


Seeker  Hardware  Unsealed 
Guidance  Command  (U/D) 


Azimuth  Seeker  Gimbal  Angle 


Elevation  Seeker  Gimbal  Angle 


Aero  Roll  Angle 


Missile  Roll  Euler  Angle 


Pitch  Carco  Command 


Carco  Table  Pitch  Feedback 


Difference  between  Carco  Pitch 
Command  and  Feedback 


Commanded  Carco  Roll  Position 


Carco  Roll  Position  Feedback 


Azimuth  Carco  Command 


Carco  Table  Yaw  Feedback 


Difference  betweenCarco  Yaw 
Command  and  Feedback 


Voltage  Command  to  Azimuth 
Mirror 


Feedback  from  Azimuth  Mirror 


Voltage  Command  to  Elevation 
Mirror 


Feedback  from  Elevation  Mirror 


Normalized,  Limited  Command 
to  the  Mirror  Translation  Axis 


Feedback  from  Mirror 
Translation  _ 


Flag  Output  by  Flare  Simulator 
Indicating  Flare  Release 


Table  C-4.  Parameters  Recorded  at  AIM-9M  SIMLAB  (Cont’d) 


Missile  Entity  State 
Data 


XM 


YM 


ZM 


XMD 


YMD 


ZMD 


RANGE 


VELOCITY 


MACH 


BETA 


ALPHA 


R 


WSSF  MS  LAT 


WSSF  MS  LON 


WSSF_MS_HDG 


WSSF_MS_VN 


WSSF_MS_VE 


WSSF  MS  VD 


Description  _ 


SIMLAB  sim  computer  output  for 
missile  _ 


Missile  Position  Along  InertialX-Axis 


Missile  Position  Along  InertialY-Axis 


Missile  Position  Along  InertialZ-Axis 


Missile  Velocity  Along  InertialX-Axis 


Missile  Velocity  Along  InertialY-Axis 


Missile  Velocity  Along  InertialZ-Axis 


Missile  Accel  Along  InertialX-Axis 


MissileAccel  Along  InertialY-Axis 


Missile  Accel  Along  InertialZ-Axis 


Distance  between  Missile  and  Target 


Total  Missile  Veloci 


Missile  Mach  Number 


Missile  Sideslip  Angle  of  Attack 


Missile  Pitch  Angle  of  Attack 


Missile  Inertial  Yaw  Rate 


Missile  Inertial  Pitch  Rate 


Missile  Inertial  Roll  Rate 


Missile  Latitude 


Missile  Longitude 


Missile  Heading,  Shooter/  Target  Earth 
Frame  _ 


Missile  North  Velocity  Component, 
Shooter/  Target  Earth  Frame 


Missile  East  Velocity  Component, 
Shooter/  Target  Earth  Frame 


Missile  Down  Velocity  Component, 
Shooter/  Target  Earth  Frame 


Units 


Feet 


Feet 


Feet 


Feet/Sec 


Feet/Sec 


Feet/Sec 


Feet 


Feet/Sec 


Radians/Sec 


Radians/Sec 


Radians/Sec 


Radians 


Radians 


Radians 


Seeker  TM  Signals 


Rate  Bias 


Lambda  Com 


AGC  Overshoot 


AGC  Control 


Solenoid  Comm  U/D 


Solenoid  Comm  R/L 


AGC  Audio 


Lambda 


Ratio  Detect 


Signals  from  Seeker  Hardware 


Feet/Sec  Every  50  ms 


Feet/Sec  I  Every  50  ms 


Feet/Sec  I  Every  50  ms 


Table  C-4.  Parameters  Recorded  at  AIM-9M  SIMLAB  (Cont’d) 


Data  Requirement 

Description 

Units 

Target  Entity  State 
Output  Data 

SIMLAB  sim  computer  output  for 
target 

Every  trial 

XT 

Target  Position  Along  Inertial  X-Axis 

Feet 

YT 

Target  Position  Along  Inertial  Y-Axis 

Feet 

ZT 

Target  Position  Along  Inertial  Z- Axis 

Feet 

Target  Velocity  Along  Inertial  X-Axis 

Feet/Sec 

Target  Velocity  Along  Inertial  Y-Axis 

Feet/Sec 

Target  Velocity  Along  Inertial  Z- Axis 

Feet/Sec 

Total  Target  Velocity 

Feet/Sec 

W  S  SF_T  GP_L  AT 

Target  Latitude  From  Conversion  of 
Inertial  Coordinates 

Radians 

Every  50  ms 

WSSF_TGP_LON 

Target  Longitude  From  Conversion  of 
Inertial  Coordinates 

Radians 

Every  50  ms 

TLAT_P 

Target  Latitude  From  Integration  of 
Target  North  Velocity 

Feet/Sec 

Every  50  ms 

TLON_P 

Target  Longitude  From  Integration  of 
Target  East  Velocity 

Feet/Sec 

Every  50  ms 

Target  Entity  State 
Input  Data 

Logged  by  SIMLAB  sim  from  WSIC 

Every  trial 

WSSF  TG  LAT 

Target  Latitude 

Radians 

WSSF  TG  LON 

Target  Longitude 

Radians 

WSSF  TG  ALT 

Target  Altitude 

Feet 

WSSF_TG_PIT 

Target  Pitch  Euler  Angle,  Shooter/ 
Target  Earth  Frame 

Radians 

Every  50  ms 

WSSF_TG_ROL 

Target  Roll  Euler  Angle,  Shooter/ 

Target  Earth  Frame 

Radians 

Every  50  ms 

WSSF_TG_HDG 

Target  Heading,  Shooter/  Target  Earth 
Frame 

Radians 

Every  50  ms 

WSSF_TG_VN 

Target  North  Velocity  Component, 
Shooter/  Target  Earth  Frame 

Feet/Sec 

Every  50  ms 

WSSF_TG_VE 

Target  East  Velocity  Component, 
Shooter/  Target  Earth  Frame 

Feet/Sec 

Every  50  ms 

WSSF_TG_VD 

Target  Down  Velocity  Component, 
Shooter/  Target  Earth  Frame 

Feet/Sec 

Every  50  ms 

Table  C-4.  Parameters  Recorded  at  AIM-9M  SIMLAB  (Concluded) 


Description 

Launch  Conditions 

Determined  by  SIMLAB  simulation 

RANGEO 

Initial  Distance  between  Missile  and 
Target 

SIGMA_AZ0 

Initial  Target  Aspect  Angle 

SIGMA.ELO 

Initial  LOS  Elevation  Angle 

ALTL 

Missile  Launch  Altitude 

VELJL 

Missile  Launch  Velocity 

ZLEAD 

Initial  Lead  Angle 

ALTT 

Initial  Target  Altitude 

VEL_T 

Initial  Target  Velocity 

FLARE  TIME 

Time  of  flare  sim  start  relative  to  launch 

INITIAL  RANGE 
RATE 

Rate  range  between  missile  and  target 
is  decreasing  at  launch 

Final  Conditions 

Determined  by  SIMLAB  simulation 

Projected  Miss 
Distance  (Horizontal) 

Horizontal  distance  of  closest  approach 
between  missile  and  target  extrapolated 
from  end  of  simulation 

Projected  Miss 
Distance  (Vertical) 

Vertical  distance  of  closest  approach 
between  missile  and  target  extrapolated 
from  end  of  sim 

TIME  OF  FLIGHT 

Duration  of  missile  flight  at  end  ofsim 

TIME_TO_GO 

Remaining  time  aftersim  stops  until 
missile  reaches  miss  distance 

FINAL  RANGE 

Distance  between  missile  and  target  at 
end  of  sim 

FINAL  RANGE 
RATE 

Rate  range  between  missile  and  target 
is  decreasing  at  end  ofsim 

PDU  Data 

Logged  on  SIMLAB  STRICOM 

Logger _ 

Missile  entity  state 
PDU 

Generated  by  SIMLAB,  see  Table  Bl-3 

Fire  (missile)  PDU 

Generated  by  SIMLAB,  see  Table  Bl-5 

Detonation  PDU 

Generated  by  SIMLAB,  see  Table  Bl-6 

Target  entity  state 
PDU 

Received  at  SIMLAB  from  WSIC,  see 
Table  B 1-2 

Fire  (flare)  PDU 

Received  at  SIMLAB  from  WSIC,  see 
Table  B 1-4 

Shooter  entity  state 
PDU 

Received  at  SIMLAB  from  WSSF,  see 
Table  Bl-1 

Feet  Each  trial,  once 


Each  trial,  once 


Feet 


Feet/Sec  Each  trial,  once 


Feet 


Feet/Sec  Each  trial,  once 


Sec 


Feet/Sec  Each  trial,  once 


Feet  I  Each  trial,  once 


Feet  Each  trial,  once 


Each  trial,  once 


Each  trial,  once 


Feet  Each  trial,  once 


Feet/Sec  I  Each  trial,  once 


Each  trial,  when 
sent 


Each  trial,  once 


Each  trial,  once 


Each  trial,  once 


Table  C-5.  Parameters  Recorded  at  TCAC 


Data  Requirement 

Description 

Units 

PDU  Data 

Logged  on  TCAC  STRICOM 
Logger 

Shooter  entity  state  PDU 

Received  at  TCAC  from 

WSSF,  see  Table  Bl-1 

Each  trial, 
upon  receipt 

Target  entity  state  PDU 

Received  at  TCAC  from 

WSIC,  see  Table  Bl-2 

Each  trial, 
upon  receipt 

Missile  entity  state  PDU 

Received  at  TCAC  from 
SIMLAB,  see  Table  Bl-3 

Each  trial, 
upon  receipt 

Fire  (flare)  PDU 

Received  at  TCAC  from 

WSIC,  see  Table  Bl-4 

Each  trial, 
once 

Fire  (missile)  PDU 

Received  at  TCAC  from 
SIMLAB,  see  Table  B 1-5 

Each  trial, 
once 

Detonation  PDU 

Received  at  TCAC  from 
SIMLAB,  see  Table  Bl-6 

Each  trial, 

once 

Data  PDU1 

Received  at  TCAC  from  Pt. 
Mugu,  used  to  drive  test 
control  displays 

Each  trial, 
upon  receipt 

Note:  (1)  The  following  test  control  display  parameters  were  contained  in  the  Data  PDU: 

-  Shooter  Altitude  (ft) 

-  Shooter  Mach  Number 

-  Target  Altitude  (ft) 

-  Target  Mach  Number 

-  Target  Acceleration  (g’s) 

-  Closing  Speed  Between  Shooter  and  Target  (knots) 

-  Data  Needed  to  Determine  Target  Aspect  Angle  (degrees) 

-  Shooter  Radar  Azimuth  and  Elevation  (degrees) 

-  Range  Between  Shooter  and  Target  (ft) 

-  Radar  Mode  Discrete 

-  Master  Arm  Discrete 

-  Trigger  Discrete 


C-9 


ANNEX  1 


PDU  DESCRIPTIONS 


TABLE  OF  CONTENTS 


Page 


Cl.  1.0  General  Specifications 

Cl-1 

Cl.  1.1  PDU  Data  Types 

Cl-1 

C 1 . 1 .2  Enumeration  Representation 

Cl-1 

C 1 . 1 .3  Number  Representation 

Cl-1 

Cl.  1.4  Coordinate  Systems 

Cl-1 

LIST  OF  TABLES 

Page 

Table  C 1  - 1 .  Shooter  Entity  State  PDU 

Cl-4 

Table  Cl -2.  Target  Entity  State  PDU 

Cl-6 

Table  Cl -3.  Missile  Entity  State  PDU 

Cl-8 

Table  Cl -4.  Fire  (Flare)  PDU 

Cl-10 

Table  Cl-5.  Fire  (Missile)  PDU 

Cl-12 

Table  Cl -6.  Detonation  PDU 

Cl-14 

LIST  OF  FIGURES 

Page 

Figure  Cl-1.  World  Coordinate  System 

Cl-2 

Figure  Cl -2.  Entity  Coordinate  System 

Cl-2 

Figure  C 1  -3 .  Definition  of  Entity  Orientation 

Cl-3 

Cl-i 


PDU  DESCRIPTIONS 


Cl.  1.0  General  Specifications  :  The  following  specifications  apply  to  all  PDU  tables  in  this  annex 
unless  otherwise  noted. 

Cl.  1.1  PDU  Data  Types  -  Data  types  are  designated  as  follows: 

E  =  Enumeration 

UI  =  Unsigned  Integer 

SI  =  Signed  Integer 

FP  =  Floating  Point  Number 

DPFP  =  Double  Precision  Floating  Point  Number 

Bool  =  Boolian  Field 

(na)  =  not  applicable 

Cl.  1.2  Enumeration  Representation  -  All  enumerated  types  shall  begin  with  “zero”  for  the  first 
element  of  the  enumeration.  Enumerations  may  have  any  size  between  1  and  32  bits.  Zero  shall 
mean  “other”. 

Cl.  1.3  Nnmhe-r  Representation  -  Numbers  shall  be  represented  as  either  floating  point  numbers 
or  integers. 

Cl. 1.3.1  Floating  Point  Numbers:  Single  and  double  precision  floating  point  numbers  shall 
adhere  to  the  IEEE  754-1985  Standard. 

Cl. 1.3.2  Integers:  Integers  shall  be  represented  as  signed  or  unsigned.  Signed  Integers  shall  be 
represented  in  2’s-complement  form  where  the  most  significant  bit  shall  designate  the  sign  bit. 
This  bit  shall  have  a  value  of  “0”  for  positive  numbers  and  “1”  for  negative  numbers  .  Integers 
may  have  a  size  of  8, 16,  or  32  bits. 

Cl. 1.4  Coordinate  Systems 

Cl.  1.4.1  World  Coordinates  -  The  shape  of  the  world  is  described  by  the  WGS-84  standard, 
DMA  TR  8350.2.  The  world  coordinate  system  is  a  right-handed,  geocentric  Cartesian  system 
(see  Figure  Cl-1).  The  origin  is  the  centroid  of  the  earth;  the  positive  X-axis  passes  through  the 
prime  meridian  at  the  equator;  the  positive  Y-axis  passes  through  90  degrees  East  longitude  at  the 
Equator;  the  positive  Z-axis  passes  through  the  North  pole. 

Cl.  1.4.2  Entity  Coordinates  -  The  entity  coordinate  system  is  a  right-handed  Cartesian  coordinate 
system  (see  Figure  Cl-2).  The  origin  is  at  the  center  of  the  entity’s  bounding  volume  (excluding 
articulated  or  attached  parts);  the  positive  X-axis  is  toward  the  front;  the  positive  Y-axis  is  to  the 
right  side;  the  positive  Z-axis  is  out  the  bottom. 


Cl-1 


Cl.  1.4.3  Location  of  Entity  -  Defined  as  the  location  of  the  origin-of-entity  in  World 
Coordinates. 

Cl.  1.4 .4  Orientation  of  Entity  -  Defined  as  three  angles  describing  the  succession  of  rotations 
needed  to  transform  from  World  Coordinate  system  to  entity’s  coordinate  system.  The  order  of 
rotation  is:  (1)  Rotate  about  Z  by  angle  psi,  (2)  rotate  about  the  new  Y  by  angle  theta,  (3) 
rotate  about  newest  X  by  the  angle  phi.  See  Figures  Cl-3athru  Cl-3c. 


Z  axis  =  Earth  axis 


Figure  Cl-1.  World  Coordinate  System 


Figure  Cl-2.  Entity  Coordinate  System 
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Z&Z' 


Figure  Cl-3a.  First  Rotate  About  Z-Axis  By  Psi 


Figure  Cl-3b.  Second  Rotate  About  Y’-Axis  By  Theta 


Figure  Cl-3c.  Third  Rotate  About  X”-Axis  By  Phi 


Figure  Cl-3.  Definition  of  Entity  Orientation 
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Table  Cl-1.  Shooter  Entity  State  PDU 
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Table  Cl-1.  Shooter  Entity  State  PDU  (Concluded) 


Note:  (1)  LSB  of  32-bit  word  is  set  to  “1”  to  indicate  use  of  absolute  (  UTC)  time  stamp.  Value 
of  the  2nd  LSB  of  the  32-bit  word  is  3600/(231  -  1)  or  1 ,67e-6  sec. 
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Table  Cl-2.  Target  Entity  State  PDU 
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r 


Table  Cl-2.  Target  Entity  State  PDU  (Concluded) 


Field 

Size 

(Bits) 

Category 

No. 

Bits 

Function 

PDL 

Data 

Units 

Enum 

320 

Dead 

Reckoning 

Parameters 

8 

- 

2 

120 

- 

- 

32 

m/sec2 

- 

- 

32 

Ent  Linear  Accel  -  Y 

m/sec2 

- 

- 

32 

Ent  Linear  Accel  -  Z 

m/sec2 

- 

- 

32 

Ent  Angular  Vel. 

rad/sec 

- 

- 

32 

Ent  Angular  Vel. 

rad/sec 

- 

- 

32 

Ent  Angular  Vel. 

rad/sec 

- 

- 

96 

i 

Entity 

Markings 

8 

Character  Set 

- 

0 

Unused 

8 

- 

0 

- 

8 

- 

0 

- 

8 

- 

0 

- 

8 

- 

0 

- 

8 

- 

0 

- 

8 

- 

0 

- 

8 

- 

0 

- 

8 

- 

0 

- 

8 

- 

0 

- 

8 

- 

0 

- 

8 

- 

0 

| 

32 

32 

- 

0 

nxl28 

Articulation 

Parameters 

8 

Param.  Type  Desig. 
Change 

ID  -  attached  to  - 
Parameter  Type 
Parameter  Value 

Not  Applicable  -  F-14D  WSIC 
generates  no  Articulation  Parameters 
for  this  Exercise 

8 

16 

32 

64 

Note:  (1)  LSB  of  32-bit  word  is  set  to  “1”  to  indicate  use  of  absolute  (  UTC)  time  stamp.  Value 
of  the  2nd  LSB  of  the  32-bit  word  is  3600/(231  -  1)  or  1 ,67e-6  sec. 
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Table  Cl -3.  Missile  Entity  State  PDU 
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Table  Cl-3.  Missile  Entity  State  PDU  (Concluded) 


Field 

Size 

(Bits) 

Category 

No. 

Bits 

Function 

PDL 

Data 

Units 

Enum 

Enumeration 

Meaning 

320 

Dead 

Reckoning 

Parameters 

8 

Dead  Reck.  Algor. 

- 

2 

120 

- 

- 

32 

Ent  Linear  Accel  -  X 

m/sec2 

- 

- 

32 

Ent  Linear  Accel  -  Y 

- 

- 

32 

Ent  Linear  Accel  -  Z 

m/sec2 

- 

- 

32 

Ent  Angular  Vel. 

rad/sec 

- 

- 

32 

Ent  Angular  Vel. 

rad/sec 

- 

- 

32 

Ent  Angular  Vel. 

rad/sec 

- 

- 

96 

Entity 

Markings 

8 

Character  Set 

- 

0 

Unused 

8 

- 

0 

- 

8 

- 

0 

- 

8 

- 

0 

- 

8 

- 

0 

- 

8 

- 

0 

- 

8 

- 

0 

- 

8 

- 

0 

- 

8 

- 

0 

- 

8 

- 

0 

- 

8 

- 

0 

- 

8 

- 

0 

wm 

32 

32 

- 

0 

nxl28 

Articulation 

Parameters 

8 

Param.  Type  Desig. 
Change 

ID  -  attached  to  - 
Parameter  Type 
Parameter  Value 

Not  Applicable  -  SIMLAB  generates 
no  Articulation  Parameters  for  this 
Exercise 

8 

16 

32 

64 

Note:  (1)  LSB  of  32-bit  word  is  set  to  “1”  to  indicate  use  of  absolute  (  UTC)  time  stamp.  Value 
of  the  2nd  LSB  of  the  32-bit  word  is  3600/(231  -  1)  or  1.67e-6  sec. 
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Table  Cl-4.  Fire  (Flare)  PDU 


Field 

Size 

(Bits) 

Category 

No. 

Bits 

Function 

PDL 

Data 

Units 

Enum 

Enumeration 

Meaning 

96 

PDU 

Header 

8 

Protocol  Version 

- 

4 

DISVer  2.0.4 

8 

Exercise  ID 

- 

1 

JADS  LSP 

8 

PDU  Type 

- 

2 

Fire 

8 

- 

2 

Warfare 

32 

1 

sec 

- 

16 

octects 

96 

PDU=96  octets 

16 

Padding 

dim 

- 

- 

48 

Firing 
Entity  ID 

16 

Site 

- 

3 

WSIC 

16 

Application 

- 

1 

F-14  Simul 

16 

Entity 

- 

1 

Target 

48 

Target 
Entity  ID 

16 

Site 

- 

2 

SIMLAB 

16 

Application 

- 

1 

AIM-9  Simul 

16 

Entity 

- 

1 

Missile 

48 

Munition  ID 

16 

Site 

- 

3 

WSIC 

16 

- 

1 

F-14  Simul 

16 

Entity 

- 

1 

Flare 

48 

Event  ID 

16 

Site 

- 

3 

WSIC 

16 

Application 

- 

1 

F-14  Simul 

16 

Entity 

- 

1 

Flare  Fire 

32 

Padding 

32 

dim 

! 

- 

192 

Location 

in 

World 

64 

X  ( WGS  84  world  coord) 

meters 

- 

- 

64 

Y  (WGS84  world  coord) 

meters 

- 

- 

64 

Z  (WGS84  world  coord) 

meters 

- 

- 
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Table  Cl-4.  Fire  (Flare)  PDU  (Concluded) 


Field 

Size 

(Bits) 

Category 

No. 

Bits 

Function 

PDU  Data  I 

Units 

Enum 

Enumeration 

Meaning 

128 

Burst 

Descr 

Mun 

Type 

8 

- 

2 

Munition 

8 

Domain 

“ 

3 

Anti-Guided 

Munition 

16 

Country 

- 

225 

United  States 

8 

Category 

- 

2 

Ballistic 

8 

- 

0 

wmmMm 

8 

- 

0 

■BH 

8 

Extra 

- 

0 

(other) 

16 

Warhead 

- 

3000 

Illumination 

16 

Fuze 

- 

2300 

Timed,  pyrotech 

16 

Quantity 

rounds 

- 

- 

16 

Rate 

■a 

- 

- 

96 

Velocity 

32 

X  (WGS84  world  coord) 

m/sec 

- 

- 

32 

Y  (WGS84  world  coord) 

m/sec 

- 

- 

32 

Z  (WGS84  world  coord) 

m/sec 

- 

- 

32 

32 

meters 

- 

“ 

Note:  (1)  LSB  of  32-bit  word  is  set  to  “1”  to  indicate  use  of  absolute  (  UTC)  time  stamp.  Value 
of  the  2nd  LSB  of  the  32-bit  word  is  3600/(231  -  1)  or  1 .67e-6  sec. 


* 
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Table  Cl-5.  Fire  (Missile)  PDU 


Field 

Size 

(Bits) 

Category 

No. 

Bits 

Function 

PDL 

I  Data  | 

Units 

Enum 

96 

PDU 

Header 

8 

Protocol  Version 

- 

4 

DIS  Ver  2.0.4 

8 

Exercise  ID 

- 

1 

JADS  LSP 

8 

PDU  Type 

- 

2 

Fire 

8 

- 

2 

Warfare 

32 

!sn  ■ 

sec 

- 

16 

octects 

96 

PDU=96  octets 

16 

Padding 

dim 

- 

- 

48 

Firing 
Entity  ID 

16 

Site 

- 

1(2) 

WSSF 

16 

Application 

- 

1 

AIM-9  Simul 

16 

Entity 

- 

2  (2) 

Missile 

48 

Target 
Entity  ID 

16 

Site 

- 

3 

WSIC 

16 

- 

1 

F-14  Simul 

16 

Entity 

- 

1 

Target 

48 

Munition  ID 

16 

Site 

- 

2 

SIMLAB 

16 

- 

1 

AIM-9  Simul 

16 

Entity 

- 

1 

Missile 

48 

Event  ID 

16 

Site 

- 

1(2) 

WSSF 

16 

Application 

- 

1 

AIM-9  Simul 

16 

Entity 

- 

2  (2) 

Missile  Launch 

32 

Padding 

32 

dim 

- 

- 

192 

Location 

in 

World 

64 

X  (WGS84  world  coord) 

meters 

- 

- 

64 

Y  (WGS84  world  coord) 

meters 

- 

- 

64 

Z  (WGS84  world  coord) 

meters 

- 

- 
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Table  Cl-5.  Fire  (Missile)  PDU  (Concluded) 


Field 

No. 

PDL 

Data 

Size 

(Bits) 

Category 

Bits 

Function 

Units 

Enum 

8 

- 

2 

Munition 

8 

Domain 

- 

1 

Anti-Air 

Mun 

16 

Country 

- 

225 

United  States 

Type 

8 

Category 

- 

1 

Guided 

Burst 

8 

- 

1 

AIM-9  Sidewinder 

128 

Descr 

8 

- 

5 

AIM-9R 

8 

Extra 

- 

0 

16 

Warhead 

- 

1000 

16 

Fuze 

- 

3000 

Proximity 

16 

Quantity 

rounds 

- 

- 

16 

Rate 

rpm 

- 

- 

32 

X  (WGS84  world  coord) 

m/sec 

- 

- 

96 

Velocity 

32 

Y  (WGS84  world  coord) 

m/sec 

- 

- 

32 

Z  (WGS84  world  coord) 

m/sec 

- 

- 

32 

■m 

32 

meters 

- 

- 

Notes:  (1)  LSB  of  32-bit  word  is  set  to  “1”  to  indicate  use  of  absolute  (  UTC)  time  stamp.  Value 
of  the  2nd  LSB  of  the  32-bit  word  is  3600/(231  -  1)  or  1 ,67e-6  sec. 

(2)  Although  the  WSSF  initiates  missile  launch,  the  time  of  missle  “fire”,  for  data 
acquistion  purposes  shall  be  recorded  at  the  time  of  actual  missile  separation.  These 
data  are  generated  by  the  SIMLAB  vice  the  WSSF.  However,  in  order  to  cause  the 
data  to  appear  as  if  the  WSSF  (Site  =1)  “launches”  the  missile  (Site=2),  the  WSSF 
Site  ID  (1)  is  assigned  to  the  SIMLAB  for  the  Fire  PDU.  Likewise,  the  SIMLAB 
Site  ID  (2)  is  assigned  to  the  “Entity”  and  the  “Event”  to  encode  an  appearance  that 
the  WSSF  “fired”  the  missile. 
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Table  Cl-6.  Detonation  PDU 


Field 

Size 

(Bits) 

Category 

No. 

Bits 

Function 

PDU  Data  I 

Units 

Enum 

Enumeration 

Meaning 

96 

PDU 

Header 

8 

Protocol  Version 

- 

4 

DIS  Ver  2.0.4 

8 

Exercise  ID 

- 

1 

JADS  LSP 

8 

PDU  Type 

- 

3 

Detonation 

8 

msBSESssam 

- 

2 

Warfare 

32 

SBI 

sec 

- 

16 

octects 

96 

PDU=96  octets 

16 

Padding 

- 

- 

48 

Firing 
Entity  ID 

16 

Site 

- 

2 

SIMLAB 

16 

Application 

- 

1 

AIM-9  Simul 

16 

Entity 

- 

1 

Missile 

48 

Target 
Entity  ID 

16 

Site 

- 

3 

WSIC 

16 

Application 

- 

1 

F-14  Simul 

16 

Entity 

- 

1 

Target 

48 

Munition  ID 

16 

Site 

- 

2 

SIMLAB 

16 

Application 

- 

1 

AIM-9  Simul 

16 

Entity 

- 

1 

Missile 

48 

Event  ID 

16 

Site 

- 

2 

SIMLAB 

16 

Application 

- 

1 

AIM-9  Simul 

16 

Entity 

- 

1 

W/H  Detonate 

96 

Velocity 

32 

X  (WGS  84  world  coord) 

m/sec 

- 

- 

32 

Y  (WGS84  world  coord) 

m/sec 

- 

- 

32 

Z  (WGS  84  world  coord) 

m/sec 

- 

- 

192 

Location 

in 

World 

64 

X  (WGS84  world  coord) 

meters 

- 

- 

64 

Y  (WGS84  world  coord) 

meters 

- 

- 

64 

Z  (WGS84  world  coord) 

meters 

- 

- 
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Table  Cl-6.  Detonation  PDU  (Concluded) 


Field 

No. 

PDU  Data  I 

Size 

(Bits) 

Category 

Bits 

Function 

Units 

Enum 

Enumeration 

Meaning 

8 

essssshih 

- 

2 

Munition 

8 

Domain 

- 

1 

Anti-Air 

Mun 

16 

Country 

- 

225 

United  States 

Type 

8 

Category 

- 

1 

Guided 

Burst 

8 

- 

1 

AIM-9  Sidewinder 

128 

Descr 

8 

- 

5 

AIM-9R 

8 

Extra 

- 

0 

hsshi 

16 

Warhead 

- 

1000 

16 

Fuze 

- 

3000 

Proximity 

16 

Quantity 

rounds 

- 

1  round 

16 

Rate 

rpm 

- 

- 

Location  in 

32 

X  (target  coordinates) 

- 

- 

96 

Entity 

32 

Y  (target  coordinates) 

- 

- 

Coordinates 

32 

Z  (target  coordinates) 

- 

- 

8 

Detonation 

Results 

32 

2 

Entity  Proximity 
Detonation 

8 

#  Artie.  Param 

8 

- 

- 

16 

Padding 

16 

- 

- 

8 

nxl28 

Articulation 

8 

Change 

Not  Applicable  -  SIMLAB 

Parameters 

16 

ID  -  attached  to  - 

Missile/Warhead  does  not  generate 

32 

Articulation  Parameters 

64 

Parameter  Value 

1 

Note:  (1)  LSB  of  32-bit  word  is  set  to  “1”  to  indicate  use  of  absolute  (  UTC)  time  stamp.  Value 
of  the  2nd  LSB  of  the  32-bit  word  is  3600/(231  -  1)  or  1.67e-6  sec. 
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D.1.0  SCOPE 


D.1.1  IDENTIFICATION 

The  Joint  Advanced  Distributed  Simulation  (  JADS)  Program  is  chartered  to  investigate  the  utility 
of  using  Advanced  Distributed  Simulation  (ADS)  as  a  methodology  for  both  developmental  and 
operational  test  and  evaluation,  as  well  as  identifying  growth  requirements  needed  to  better  meet 
the  needs  of  the  test  and  evaluation  (  T&E)  community.  JADS  Joint  Test  and  Evaluation  (  JT&E) 
is  the  implementation  of  this  investigation. 

As  currently  approved,  JADS  JT&E  includes  two  tests:  (1)  A  System  Integration  Test  (SIT)  and 
(2)  an  End-to-End  ( ETE)  test.  The  SIT  examines  the  use  of  ADS  technology  to  support  air-to-air 
missile  testing.  The  ETE  examines  the  use  of  ADS  technology  to  support  C4I  system  testing. 
The  SIT  is  implemented  in  two  phases:  (1)  A  Live  Fly  Phase  and  (2)  a  Linked  Simulators  Phase 
(LSP). 

This  is  the  Interface  Control  Document  (  ICD)  for  the  LSP  of  the  JADS  JT&E  program.  The  LSP 
includes  three  test  missions:  (1)  A  Verification  and  Validation  (V&V)  Mission  (Mission  #1),  (2)  a 
Parametric  Mission  (Mission  #2),  and  (3)  a  Data  Latency  Mission  (Mission  #3). 

D.1.2  SYSTEM  OVERVIEW 

The  LSP  test  configuration,  shown  in  Figure  D-l,  is  comprised  of  the  F/A-18  Weapon  System 
Support  Facility  (WSSF)  and  the  AIM-9  hardware-in-the-loop  Simulation  Laboratory  (SIMLAB) 
both  located  at  Naval  Air  Warfare  Center  (NAWCWPNS),  China  Lake,  CA;  the  F-14D  Weapons 
System  Integration  Center  (WSIC)  and  the  Battle  Management  Interoperability  Center  (BMIC) 
both  located  at  NAWCWPNS,  Point  Mugu,  CA;  and  the  JADS  Test  Control  and  Analysis  Center 
(TCAC)  located  at  Kirtland  AFB,  NM.  These  facilities  are  networked  via  a  subset  of  the 
NAWCWPNS  Real-Time  Network  (NRNet)  as  shown  in  Figure  D-2  and  Tables  D-l  and  C-2. 
All  closed-loop  simulation  data  flow  between  these  facilities  is  via  standard  Distributed  Interactive 
Simulation  (DIS)  Protocol  Data  Units  (PDUs),  Version  2.0.4,  transmitted  over  this  network.  The 
network  terminates  in  a  Network  Interface  Unit  (NIU)  at  each  facility.  The  NIU  interfaces  with  a 
Simulation  Interface  Unit  (  SIU)  which,  in  turn  interfaces  with  the  high-fidelity  simulation 
applications  running  autonomously  at  each  facility  as  shown  in  Figure  D-3.  The  aggregate 
NIU/SIU  hardware/software  systems  at  each  facility  implement  all  required  communications 
functions  defined  by  the  OSI  7-Layer  Reference  Model.  In  addition  to  the  DIS  network,  a  MIL- 
STD-1553  Networking  System  (MNS-1)  link,  which  is  part  of  the  NRNet,  also  connects  the 
WSSF  with  the  SIMLAB  for  the  singular  purpose  of  transmitting  prelaunch  missile  preparation 
information  from  the  WSSF  to  the  SIMLAB.  The  facilities  are  also  linked  by  STU-III  voice 
communications . 

The  detailed  network  hardware  diagram  for  the  LSP  is  given  in  Annex  2. 
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Figure  D-l.  JADS  LSP  Configuration 
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Figure  D-2.  NRNet  Block  Diagram 


D-3 


Table  D-l.  NRNet  IP  Address  List 


199.208.229.146  NRNet  Cisco  Router  Serial  0-LRCC,  LN  Router _ Bemie  Orleit _ (619)  939-6980 

199.208.228.145  NRNet  Cisco  Router  Ethernet  0  .146 -.158  Ed  Slack  (619)927-1457 

.159  _ 
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Table  D-2.  LSP  Router  Hardware  and  Software 


Location 

Laboratory 

Router 

Hardware 

Router 

Software 

Remarks 

Point  Mugu 

RCCO 
(Bldg.  531) 

Bay  Networks  (Wellfleet) 

CN  Router 

Version  7.80,  Fix  7 

wise 

Cisco  2501 

RCCO 
(Bldg.  531) 

IDNX-20 

TBD 

JADS  KAFB  WAN 
Interface 

China  Lake 

LRCC 

(Bldg.  31455) 

Bay  Networks  (Wellfleet) 

LN  Router 

Version  5.77 

WSSF 

Cisco  2501 

Version  10.3  (12) 

SIMLAB 

Cisco  2501 

Version  10.3  (7) 

Figure  D-3.  Network  Configuration  for  LSP  Simulation  Exercises 
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D.1.3  DOCUMENT  OVERVIEW 


This  ICD  applies  only  to  the  LSP  of  the  JADS  JT&E  and  describes  only  those  interface 
requirements  on  the  “network  side”  of  the  data  entry/exit  point  of  the  NIU  (or  equivalent)  located 
at  each  facility  site.  Any  interface  requirements  of  the  NIU  and/or  SIU  are  beyond  the  scope  of 
this  document.  Additionally,  this  ICD  does  not  address  the  interface  requirements  related  to  any 
simulation  exercise  beyond  those  described  in  the  Test  Activity  Plan  for  thdADS  LSP. 

Standard  DIS  terminology,  as  defined  in  the  Standard  for  Distributed  Interactive  Simulation  - 
Application  Protocols,  Version  2.0  Fourth  Draft,  is  used  throughout  this  document  to  describe 
elements,  processes,  functions,  etc. 
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D.2.0  APPLICABLE  DOCUMENTS 

D.2.1  GOVERNMENT  DOCUMENTS 
None 

D.2.2  'NON-  GOVERNMENT  DOCUMENTS 

The  following  documents  to  the  issue  shown  form  a  part  of  this  specification  to  the  extent 
specified  herein.  In  the  event  of  conflict  between  the  documents  referenced  herein  and  contents  of 
this  specification,  the  contents  of  this  specification  shall  be  considered  a  superseding  requirement. 

Standard  for  Distributed  Interactive  Simulation  --  Application  Protocols  Version  2.0  Fourth 
Draft,  Institute  for  Simulation  and  Training,  University  of  Central  Florida,  February  1994 

This  document  is  available  from: 

Institute  for  Simulation  and  Training 
3280  Progress  Drive 
Orlando,  FL  32862 

IEEE  754-1985-  IEEE  Standard  for  Binary  Floating  Point  Arithmetic,  IEEE  Product  #SH101 16 

This  document  is  available  from: 

IEEE  Inc. 

445  Hoes  Lane 
P.O.  Box  1331 

Piscataway,N.J.  08855-1331 
Telephone:  1-800-678-IEEE 

ISO  7498-  Information  Processing  Systems  -  Open  Systems  Interconnection  -  Basic  Reference 
Model 


This  document  is  available  from: 

American  National  Standards  Institute  (ANSI) 
Sales  Department 
1 1  West  42nd  Street 
New  York,  NY  10036 
Telephone:  212-642-4900 
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IST-CR-95-14  Enumeration  and  Bit-encoded  Values  for  use  with  IEEE  1278.1-1995,  Standard 
for  Distributed  Interactive  Simulation  Protocols 

This  document  is  available  from: 

University  of  Central  Florida 
Center  for  Continuing  Education 
Orlando,  FL  32816-0177 
Telephone:  407-249-6100 

Technical  society  and  technical  association  specifications  and  standards  are  generally  available  for 
reference  from  libraries.  They  are  also  distributed  among  technical  groups  and  using  Federal 
Agencies. 
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D.3.0  INTERFACE  SPECIFICATION 


D.3.1  INTERFACE  DIAGRAMS 

Figure  D-4  illustrates  the  functional  data  flow  between  each  of  the  simulation  entities.  In  all  cases 
but  one,  data  is  transmitted  via  standardized  DIS  PDUs  (Version  2.0.4).  The  one  exception  is 
prelaunch  missile  preparation  initial  conditions  data  that  are  generated  in  the  WSSF  and 
transmitted  to  the  missile  in  the  SIMLAB  via  an  MNS-1  real-time  network  using  TCP/IP  protocol 
over  Ethernet.  Detailed  contents  and  formats  of  each  PDU  are  found  in  Appendix  C.  Content 
and  format  of  the  MNS-1  interface  is  found  in  Annex  1  to  this  appendix. 


Figure  D-4.  Functional  Data  Flow  Diagram  for  LSP  Simulation  Exercise 
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Origin,  sequence,  and  utilization  of  PDUs  are  shown  in  Figur<£)-5. 


SIMULATION  ENTITIES 


F-18W55F 

(Shorter) 


M4DW5IC 
(Target,  warning 
System,  and  Flare) 


TsT  Control  an  d  Analyse  Center  (TCAC)  Esues 
"StarT  command  via  Telecon  to  the  3 
Simulators  (W55F,  W5IQ  and  SIMLA B) . 

simulators  each  acknowledge  receipt  of  the 
"StarT command  vid  Telecon. 

F-18  (5hootel)  andF-lfl  (Target)  ore  initialed  at, 
a  predetermined  location  and  state  in  World 
Coordinates;  Mcsie  e  on  board  the  Shooter 
Flare  e  onboard  the  Target.  Shooter  and  Target 
maneuver  to  Tiring’  geometry. 

(Mfesilefe  propped  via  non-Dis  interface.) 


Mfeiie  e  tired  at  predetermined  launch  point 
and  proceed  tofly  out  from  5  hooter  towards 
Targrt. 

Target  senses  mksile  launch  and  at  pre 
determined  the  after  launch,  beghs  evurve 
maneuver  and  initiates  Flare  deployment. 


SIM  LAB  reports  detonation/lack  of  detonation 
on  eitherthe  Targrt  or  Flare  or  neither. 

TCAC  ksues  ’Stop”  command  via  Teleconto3 
Simulators. 

3  Simulators  ackowledge,‘Stopf'  command  via 
Telecon. 


NOTES;  Nomenclature  within  CZ )  denotes  type  of  PDU;  "TaT  of  atow  the  origin  ofthePDU,  an  d"H  ead"  of  arrow  "Users"  of  the  PD  U. 

£3  Functions  of  Starts  op  /Acknowledge  implemented  via  Telecan  vice  PDUS. 

Figure  D-5.  Origin,  Sequence,  and  Utilization  of  Protocol  Data  Units  (PDUs)  for  LSP  Tests 


Each  facility  site  (“simulator”)  generates  one  or  more  “Simulation  Entities”,  as  defined  in  DIS 
Standard  Version  2.0.4,  and  are  described  as  follows: 

F-14D  Weapons  System  Integration  Center  (WSIC) 

The  F-14D  WSIC  is  a  full,  6-degree-of-ffeedom,  real-time,  closed-loop,  hardware-in-the-loop 
simulation  laboratory  that  incorporates  actual  F-14D  avionics,  controls  and  displays,  and  tactical 
software  in  a  1:1  mockup  of  the  forward  module  of  the  F-14D.  The  WSIC  generates  two 
simulation  entities  for  the  LSP  application  exercise:  (1)  the  “target”  against  which  the  missile  is 
fired  and  (2)  a  “warning  receiver”  that  detects  that  the  target  has  been  fired  upon  and  cues  the 
expenditure  of  countermeasures  (a  “flare”). 
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AIM-9  Simulation  Laboratory  /SIMLAB) 


The  SIMLAB  is  a  6-degree-of-freedom,  real-time,  closed-loop,  hardware-in-the-loop  simulation 
laboratory  that  incorporates  actual  missile  guidance  and  control  hardware  and  software.  The 
SIMLAB  is  capable  of  incorporating  various  missile  systems,  but  for  the  LSP  exercises  will 
incorporate  an  AIM-9  Sidewinder  missile.  The  SIMLAB  is  also  capable  of  generating  computer- 
controlled  target,  clutter,  and  counter-measures  signatures.  For  the  LSP  the  SIMLAB  generates 
two  simulation  entities:  (1)  the  missile  that  is  fired  against  the  “target”  (WSIC)  by  the  F/A-18 
“shooter”  (WSSF)  and  (2)  the  characteristics  of  the  “flare”  deployed  by  the  “target”  against  the 
missile. 

F/A-18  Weapon  System  Support  Facility  (WSSF) 

The  WSSF  is  similar  to  the  WSIC  in  both  function  and  capabilities.  The  primary  difference  is  the 
WSSF  avionics  and  controls  and  displays  are  not  mounted  in  an  aircraft  mockup,  but  they  are 
mounted  in  conventional  laboratory  equipment  bays  and/or  workstations.  The  WSSF  generates 
one  simulation  entity:  the  F/A-18  “shooter.” 

Test  Control  and  Analysis  Center  (TCAC) 

The  TCAC,  or  the  BMIC,  provides  overall  simulation  exercise  control  and  monitoring  and  does 
not  generate  any  “simulation  entity”  in  terms  of  an  active  player  in  the  simulated  scenario.  Test 
control  and  monitoring  of  LSP  Mission  #1  will  be  exercised  from  the  BMIC.  Test  control  and 
monitoring  of  LSP  Missions  #2  and  #3  will  be  exercised  from  the  TCAC.  References  in  this 
document  to  the  TCAC  also  apply  to  the  BMIC  when  the  BMIC  is  used  for  test  control. 

PIS  Protocol 


DIS  Version  2.0.4  defines  27  PDU  types  organized  into  6  protocol  families.  In  general,  a  PDU  is: 

(a)  a  packet  of  application  data  containing  (1)  identification  of  information  exchanged, 
(2)  structured  format  of  the  information,  (3)  circumstances  under  which  the 
information  is  transmitted,  (4)  action  required  upon  receipt  (if  any),  (5)  state  variable 
updates,  and  (6)  constraints  governing  the  exchange  of  information. 

(b)  a  pointer  to  databases  contained  in  each  simulation  application. 

(c)  required  for  interoperability 

The  LSP  simulation  exercises  require  utilization  of  only  3  of  the  27  PDUs  from  2  of  the  6  families. 
They  are: 
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Entity  Information/Interaction  (family) 


Entity  State  PDU 

Warfare  (family) 

Fire  PDU 
Detonation  PDU 

D.3.2  F-14  WSIC  INTERFACE  SPECIFICATIONS 

The  F-14  WSIC  shall  generate  the  target  and  warning  system  simulation  entities. 

D.3.2. 1  F-14  WSIC  INTERFACE  REQUIREMENTS 

All  F-14D  WSIC  software  elements  run  autonomously  and  concurrent  with  the  software  elements 
of  the  TCAC,  WSSF,  and  SIMLAB.  Synchronization  protocol  for  supporting  concurrent 
operation  is  embedded  in  the  NIU  software  at  the  WSIC  and  is  beyond  the  scope  of  this 
document. 

DIS  PDUs  will  be  used  to  interface  the  F-14D  WSIC  with  the  network.  Specifically,  Entity  State 
(target)  and  Fire  (flare)  PDUs  will  be  generated  by  the  WSIC.  Fire  (missile),  Detonation,  and 
Entity  State  (missile  and  shooter)  PDUs  will  be  received  by  the  WSIC. 

D.3.2.2  F-14  WSIC  DATA  REQUIREMENTS 

Appendix  C,  Annex  1  lists  the  details  of  the  PDU’s  output  from  the  F-14D  WSIC  to  the  network. 
They  are: 


Table  Cl-2  F-14D  WSIC  Entity  State  (Target)  PDU 
Table  Cl -4  F-14D  WSIC  Fire  (Flare)  PDU 

Table  D-3  lists  the  PDU  inputs  required  by  the  F-14D  WSIC  from  the  network. 


Table  D-3.  PDU  Inputs  to  F-14D  WSIC 


PDU  Input  to  F-14D 

From 

Reference 

Entity  State  (shooter) 

F/A-18  WSSF 

Table  Cl-1 

Fire  (missile) 

SIMLAB 

Table  Cl -5 

Detonation 

SIMLAB 

Table  Cl-6 

Entity  State  (missile) 

SIMLAB 

Table  Cl -3 
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D.3. 2.2.3  Data  Encryption 

All  PDU  data  from  the  WSIC  NIU  to  the  network  shall  be  encrypted  before  being  transmitted  to 
the  network.  All  PDU  data  received  from  the  network  shall  be  decrypted  before  being 
transmitted  to  the  NIU. 

UTC  time  in  IRIG-B  format  will  be  used  to  time  stamp  all  data  recorded  at  the  WSIC. 

D.3.3  SIMLAB  INTERFACE  SPECIFICATIONS 

The  SIMLAB  shall  generate  the  missile  simulation  entity  and  flare  characteristics. 

D.3.3. 1  SIMLAB  INTERFACE  REQUIREMENTS 

All  SIMLAB  software  elements  run  autonomously  and  concurrent  with  the  software  elements  of 
the  TCAC,  WSSF,  and  WSIC.  Synchronization  protocol  for  supporting  concurrent  operation  is 
embedded  in  the  NIU  software  at  the  SIMLAB  which  is  beyond  the  scope  of  this  document. 

DIS  PDUs  will  be  used  to  interface  the  SIMLAB  with  the  network.  Specifically,  Entity  State 
(missile),  Fire  (missile)  and  Detonation  PDUs  will  be  generated  by  the  SIMLAB.  Fire  (flare),  and 
Entity  State  (shooter  and  target)  PDUs  will  be  received  by  the  SIMLAB. 

MIL-STD-1553  message  traffic  supported  by  TCP/IP  protocol  will  be  received  by  the  SIMLAB 
from  the  F/A-18  WSSF  via  the  MNS-1  real-time  network.  This  message  traffic  is  used 
exclusively  for  sending  missile  prelaunch  preparation  information  from  the  shooter  to  the  missile 
and  receiving  the  missile  response  during  the  missile  launch-to-eject  (  LTE)  cycle.  Detailed 
specifications  for  this  message  traffic  are  contained  in  Annex  1  to  this  appendix. 

D.3.3.2  SIMLAB  DATA  REQUIREMENTS 

D.3.3.2.1  Data  Recording  Requirements 

Appendix  C,  Table  C-4  lists  the  SIMLAB  data  recording  requirements.  These  data  are  recorded 
locally  at  the  SIMLAB  facility. 

D.3.3.2.2  SIMLAB  PDU  Descriptions 

Appendix  C,  Annex  1  lists  the  details  of  the  PDU’s  output  from  the  SIMLAB  to  the  network. 
They  are: 

Table  Cl-5  SIMLAB  Fire  (missile)  PDU 
Table  Cl -3  SIMLAB  Entity  State  (missile)  PDU 
Table  Cl -6  SIMLAB  Detonation  (missile)  PDU 

Table  D-4  lists  the  PDU  inputs  required  by  the  SIMLAB  from  the  network. 
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Table  D-4.  PDU  Inputs  to  SIMLAB 


PDU  Input  to  SIMLAB 

From 

Reference 

Entity  State  (shooter) 

F/A-18  WSSF 

Table  Cl-1 

Fire  (flare) 

F-14D  WSIC 

Table  Cl -4 

Entity  State  (target) 

F-14D  WSIC 

Table  Cl-2 

D. 3.3.2. 3  Data  Encryption 

All  PDU  data  from  the  SIMLAB  NIU  to  the  network  shall  be  encrypted  before  being  transmitted 
to  the  network.  All  PDU  data  received  from  the  network  shall  be  decrypted  before  being 
transmitted  to  the  NIU. 

UTC  time  in  IRIG-B  format  will  be  used  to  time  stamp  all  data  recorded  at  the  SIMLAB. 

D.3.4  F/A-18  WSSF  INTERFACE  SPECIFICATIONS 
The  F/A-18  WSSF  shall  generate  the  shooter  entity. 

D.3.4.1  F/A-18  WSSF  INTERFACE  REQUIREMENTS 

All  F/A-18  WSSF  software  elements  run  autonomously  and  concurrent  with  the  software 
elements  of  the  TCAC,  WSIC,  and  SIMLAB.  Synchronization  protocol  for  supporting 
concurrent  operation  is  embedded  in  the  NIU  software  at  the  WSSF  which  is  beyond  the  scope  of 
this  document. 

DIS  PDUs  will  be  used  to  interface  the  F/A-18  WSSF  with  the  network.  Specifically,  Entity  State 
(shooter)  PDUs  will  be  generated  by  the  WSSF  and  Fire  (missile  and  countermeasures),  Entity 
States  (missile  and  target),  and  Detonation  PDUs  will  be  received  by  the  WSSF. 

MIL-STD-1553  message  traffic  supported  by  TCP/IP  protocol  will  be  transmitted  via  the  MNS-1 
real-time  network  from  the  F/A-18  WSSF  to  the  SIMLAB.  This  message  traffic  is  used 
exclusively  for  sending  missile  prelaunch  preparation  information  from  the  shooter  to  the  missile 
and  receiving  the  missile  response  during  the  missile  LTE  cycle.  Detailed  specifications  for  this 
message  traffic  is  contained  in  Annex  1  to  this  appendix. 

D. 3.4.2  F/A-18  WSSF  DATA  REQUIREMENTS 

D.3.4.2.1  Data  Recording  Requirements 

Appendix  B,  Table  B-2  lists  the  F/A-18  WSSF  data  recording  requirements.  These  data  are 
recorded  locally  at  the  WSSF  facility. 
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D.3.4.2.2  F/A-18  WSSF  PDU  Descriptions 


Appendix  C,  Annex  1,  Table  Cl-1  lists  the  detailed  PDU  output  from  the  F/A-18  WSSF  to  the 
network. 

Table  D-5  lists  the  PDU  inputs  required  by  the  F/A-18  WSSF  from  the  network. 


Table  D-5.  PDU  Inputs  to  F/A-18  WSSF 


PDU  Input  to  F/A-18 
WSSF 

From 

Reference 

Entity  State  (target) 

F-14DWSIC 

Table  Cl -2 

Fire  (flare) 

F-14D  WSIC 

Table  Cl-4 

Entity  State  (missile) 

SIMLAB 

Table  Cl -3 

Fire  (missile) 

SIMLAB 

Table  Cl -5 

Detonation 

SIMLAB 

Table  Cl -6 

D.3.4.2.3  Data  Encryption 

All  PDU  data  from  the  WSSF  NIU  to  the  network  shall  be  encrypted  before  being  transmitted  to 
the  network.  All  PDU  data  received  from  the  network  shall  be  decrypted  before  being 
transmitted  to  the  NIU. 

UTC  time  in  IRIG-B  format  will  be  used  to  time  stamp  all  data  recorded  at  the  WSSF. 

D.3.5  TCAC  INTERFACE  SPECIFICATIONS 
D.3.5.1  TCAC  INTERFACE  REQUIREMENTS 

The  TCAC  shall  provide  overall  test  coordination,  control  and  monitoring. 

All  TCAC  software  applications  run  autonomously  and  concurrent  with  the  software  applications 
of  the  WSSF,  WSIC,  and  SIMLAB.  Synchronization  protocol  for  supporting  concurrent 
operation  is  embedded  in  the  NIU  software  at  TCAC.  This  protocol  is  not  within  the  scope  of 
this  document. 

DIS  PDUs  will  be  used  to  interface  the  TCAC  with  the  network.  Specifically,  Entity  State,  Fire 
and  Detonation  PDUs  will  be  received  by  the  TCAC  from  the  network. 
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D.3.5.2  TCAC  DATA  REQUIREMENTS 
D.3.5.2. 1  Data  Recording  Requirements 

Appendix  C,  Table  C-5  lists  the  TCAC  data  recording  requirements.  These  data  are  recorded 
locally  at  the  TCAC  facility. 

D.3. 5.2.2  TCAC  PDU  Descriptions 

TCAC  does  not  generate  PDU  outputs  for  the  LSP  missions. 

Table  D-6  lists  the  PDU  inputs  required  by  the  TCAC  from  the  network. 


Table  D-6.  PDU  Inputs  to  TCAC 


PDU  Input  to  TCAC 

From 

Reference 

Entity  State  (shooter) 

F/A-18  WSSF 

Table  Cl-1 

Entity  State  (target) 

F-14DWSIC 

Table  Cl -2 

Entity  State  (missile) 

SIMLAB 

Table  Cl-3 

Fire  (flare) 

F-14D  WSIC 

Table  Cl -4 : 

Fire  (missile) 

SIMLAB 

Table  Cl -5 

Detonation 

SIMLAB 

Table  Cl-6 

D.3.5.2. 3  Data  Encryption 

PDU  data  transmitted  from  the  TCAC  shall  be  encrypted  to  facilitate  data  processing  consistency 
of  all  received  PDU  traffic  at  other  facility  sites  on  the  network.  All  PDU  data  received  at  the 
TCAC  from  the  network  shall  be  decrypted  before  being  transmitted  to  the  TCAC  NIU. 

D.3.5.2.4  UTC  time  in  IRIG-B  format  will  be  used  to  time  stamp  all  data  recorded  at  the  TCAC. 
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ANNEX  1 


MNS-1  INTERPACE  DESCRIPTION 


F/A-18  WSSF  MIL-STD-1553  NETWORKING  SYSTEM  (MNS-1) 


This  appendix  defines  the  message  traffic  used  to  communicate  between  two  or  more  MNS-1 
real-time  network  interfaces.  The  MNS-ls  are  used  by  the  F/A-18  Weapon  System  Support 
Facility  (WSSF)  to  virtually  extend  the  F/A-18  MIL-STD-1553  Avionics  Bus  over  long  distances 
allowing  avionics  subsystems  to  be  remotely  located.  The  MNS-1  located  in  the  F/A-18  WSSF  is 
identified  as  the  Local  node,  while  one  or  more  MNS-ls  located  at  other  sites  are  identified  as 
Remote  nodes.  Data  is  transferred  between  the  Local  and  Remote  MNS-1  systems  using  TCP/IP 
protocol  over  Ethernet.  All  data  words  contain  16-bits.  Each  MNS-1  message  is  preceded  with  a 
two-word  header  which  contains  a  -1  followed  by  a  1,  and  a  two-word  trailer  which  contains  a  -1 
followed  by  a  2.  The  following  describe  all  the  messages  used  to  connect,  initialize,  coordinate, 
and  pass  data  between  sites. 

Dl.1.0  EOCAL-TO-REMOTE  MESSAGES 

D 1 . 1 . 1  Command  Built-In  Test  (BIT)  Message 

Word  1  -  Message  ID  =  3 

Dl.1.2  Status  Request  Message 

Word  1  -  Message  ID  =  4 

Dl.1.3  Initialize/Setup  Message 

Word  1  -  Message  ID  =  6 

Word  2  -  Number  of  1553  Message  Groups  (RT/SAs)  to  follow 


Word  3  -  Remote  MNS-1  Node  Number 

Word  4-  1553  Remote  Terminal  (RT)  Number 

Word  5  -  1553  Subaddress  (SA)  Number 

Word  6-  1553  RT/SA  Word  Count 

Word  7  -  1553  RT/SA  Communication  Rate 

Word  8-  1553  RT  Device  Type  (Mil-Std-1553A  or  B) 


Repeat  3-8  for  each  1553  Message  Group  identified  in  Word  2 
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D 1 . 1 .4  Real-T ime  Data  Message 

Word  1  -  Message  ID  =  7 
Word  2  -  Local  Message  Count 
Word  3  -  Remote  Message  Count  Echo 
Word  4  -  Discrete  Signals  1-16 
Word  5  -  Discrete  Signals  17-32 
Word  6  -  Discrete  Signals  33-48 
Word  7  -  Discrete  Signals  49-64 

Word  8  -  Number  of  1553  Message  Groups  (RT/SAs)  to  Follow 


Word  9  -  RT/SA  Message  Type  (bcrt,  type  of  mode  code) 
Word  10  -  RT  Number 
Word  1 1  -  SA  Number 
Word  12  -  Word  Count 

Word  13+  -  Data  Words  for  this  RT/SA  Message 


Repeat  9-13+  for  each  1553  Message  Group  identified  in  Word  8 


D 1 . 1 . 5  Start  Real-T  ime  Message 

Word  1  -  Message  ID  =  9 

Dl.1.6  Halt  Real-Time  Message 

Word  1  -  Message  ID  =  10 

D 1 . 1 .7  End  Program  Message 

Word  1  -  Message  ID  =  1 1 

Dl.2.0  REMOTE-TO-LOCAL  MESSAGES 

D  1.2.1  Status  Response  Message 

Word  1  -  Message  ID  =  4 

Word  2  -  Remote  Node  Number 

Word  3  -  1553  ABI  BIT  Results  (pass/fail/untested) 

Word  4  -  Clock  Timer  BIT  Results  (pass/fail/untested) 

Word  5  -  Remote  Echo  Word 

Word  6  -  Initialized  (true/false) 

Word  7  -  Current  State  (real-time/not  real-time) 
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Dl.2.2  Real-Time  Data  Message 

Word  1  -  Message  ID  =  8 
Word  2  -  Remote  Message  Count 
Word  3  -  Local  Message  Count  Echo 
Word  4  -  Discrete  Signals  1-16 
Word  5  -  Discrete  Signals  17-32 
Word  6  -  Discrete  Signals  33-48 
Word  7  -  Discrete  Signals  49-64 

Word  8  -  Number  of  1553  Message  Groups  (RT/SAs)  to  follow 


Word  9 -RT  Number 
Word  10  -  SA  Number 

Word  1 1  -  RT/SA  Status  (rtbc  on/off,  bcrt  response/no  response) 
Word  12  -  Word  Count 

Word  13+  -  Data  Words  for  this  RT/SA  Message 


Repeat  9-13+  for  each  1553  Message  Group  identified  in  Word  8 
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ANNEX  2 


LSP  NETWORK  HARDWARE  DIAGRAM 


APPENDIX  E 


INTEGRATED  TEST  PROCEDURES 
for  the 

SYSTEM  INTEGRATION  TEST 


LINKED  SIMULATORS  PHASE 


TABLE  OF  CONTENTS 


Page 

E.l  Master  Test  Control  Checklist  El-1 

E.2  Network  Coordinator  Checklist  E2-1 

E.3  Site  Supervisor  Checklist  E3-1 

E.4  SNAP  Operator  Checklist  E4-1 

E.5  DIS  Logger  Operator  Checklist  E5-1 

E.6  Test  Cards  E6-1 

E.7  Logs  and  Quick-Look  Data  Forms  E7-1 


LINKED  SIMULATORS  PHASE 

MASTER  TEST  CONTROL  CHECKLIST 

DAY  BEFORE  TEST 

Test  Controller: _ _ 

Date: _  Lab  Time: _ to _ 

In  the  spaces  provided,  record  the  time  each  action  is  completed. 

1 .  _ Ensure  enough  supplies  for  the  entire  test  period  are  on  hand  (checklist,  test  cards, 

blank  test  controller  log  sheets,  pens  or  pencils,  scratch  paper). 

2.  _ Join  the  command  voice  conference  net  by  dialing  (619)  939-0010  (SIMLAB  Site 

Supervisor  will  make  the  first  call). 

3.  _ Conduct  communication  checks  with  each  Site  Supervisor,  Laboratory  Supervisor 

and  pilot. 

4.  _ Confirm  with  the  Network  Coordinator  that  the  support  voice  conference  network 

is  operating  and  all  participants  have  checked  in. 

5.  _ Boot  all  equipment. 

6.  _ Verify  test  control  display  clock  is  synchronized  to  tie  GPS  clock. 

7.  _ Confirm  operational  readiness  of  2-D  viewer  and  test  control  display. 

8.  _ Verify  with  Site  Supervisors  and  Network  Coordinator  that  all  equipment  at  each 

site  is  fully  functional. 

9.  _ Leave  voice  conference  net. 

10.  _ Ensure  Legacy  Supervisor  has  all  still  camera,  video  and  communications 

equipment  ready  for  setup. 

1 1 .  _ Brief  mission. 

A.  _  Discuss  equipment  readiness  and  any  repairs  necessary. 

B.  _  Establish  arrival  times  formission  personnel. 

C.  _  Review  mission  procedures  and  test  cards. 

12.  _ Coordinate  repairs  as  necessary  to  prepare  equipment  for  the  next  day  (T-23hrs. 

to  T-60  min.). 
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LINKED  SIMULATORS  PHASE 
MASTER  TEST  CONTROL  CHECKLIST 
DAY  OF  TEST  BEFORE  TEST  RUNS 


Test  Controller: _ 

Date: _  Lab  Time: _ to 

T-60  min. 


1 .  Arrive  on  station  and  open  written  logs. 

2.  Test  Controller  ensure  enough  supplies  for  the  entire  test  period  are  on  hand  (test  cards, 
checklist,  blank  test  execution  log  sheets,  pen  or  pencil,  scratch  paper). 

3.  SIMLAB  Site  Supervisor  initiate  command  voice  communication  network  by  calling  the  audio 
conference  bridge  #  (6 1 9)  93 9-00 1 0. 

4.  SIMLAB  DIS  logger  operator  initiate  support  voice  communication  network  by  calling  the 
audio  conference  bridge  #  (619)  939-6338. 

5.  Ensure  all  system  clocks  are  set  to  GMT  (UTC). 

T-60  to  T-45  min. 

6.  Test  Controller,  Site  Supervisors,  Laboratory  Supervisors  and  pilots  join  the  command  voice 
communication  network  by  calling  the  audio  conference  bridge  #  (619)  939-0010. 

7.  Network  Coordinator,  SNAP  operators  and  DIS  logger  operators  join  the  support  voice 
communication  network  by  calling  the  audio  conference  bridge  #  (619)  939-6338. 

8.  Test  Controller  confirm  operational  readiness  of  2-D  viewer  and  test  control  display. 

9.  Network  Coordinator  initialize  all  files  and  ping  each  system. 

10.  Legacy  Supervisor  ensure  all  video  equipment  is  in  place  and  operational. 

T-45  min. 


1 1 .  Test  Controller  and  Network  Coordinator  conduct  communications  checks  on  their  voice 
conference  net. 

12.  Logger  operators  report  equipment  status  to  the  Network  Coordinator. 

T-15  min. 

13.  Site  Supervisors  and  Network  Coordinator  report  to  the  Test  Controller  readiness  of  sites  and 
network  for  test. 

14.  Test  Controller  makes  go/no  go  decision. 
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LINKED  SIMULATORS  PHASE 


MASTER  TEST  CONTROL  CHECKLIST 

FOR  EACH  TEST  RUN 

Test  Controller: _ _ 

Date: _  Lab  Time: _ to _ 

1.  Execute  test  cards. 

2.  After  each  run  determine  whether  to  repeat  the  run  or  proceed  to  the  next  test  point. 

3.  Instruct  sites  to  archive  data  and  the  number  of  next  run. 

4.  Terminate  test. 
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LINKED  SIMULATORS  PHASE 
MASTER  TEST  CONTROL  CHECKLIST 
END  OF  DAY 

Test  Controller: _ _ _ 

Date: _ _  Lab  Time: _ to _ 

1  _ Direct  Network  Controller  to  start  file  transfers. 

2.  _ _  Gather  test  execution  logs  and  other  notes. 

3.  _ Leave  voice  conference  net. 

4.  _ Network  Coordinator  creates  master  data  tape. 

5.  _ Turn  off  all  equipment. 

6.  _ Conduct  mission  debrief. 

A.  Problems  encountered. 

B.  Specific  comments  on  individual  runs. 

C.  Equipment  status. 
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LINKED  SIMULATORS  PHASE 
NETWORK  COORDINATOR  CHECKLIST 
DAY  BEFORE  TEST 


Network  Coordinator: _ 

Date: _  Lab  Time: _ to _ 

In  the  spaces  provided,  record  the  time  each  action  is  completed. 

1 .  _ Ensure  enough  supplies  for  the  entire  test  period  are  on  hand  (checklist,  blank 

hardware/software  discrepancy  and  network  coordinator  log  sheets,  file  name  lists, 
pens  or  pencils,  scratch  paper,  4mm  tape  cartridges). 

2.  _ Join  the  support  voice  conference  net  by  dialing  (619)  939-6338  (SIMLAB 

operator  will  make  the  first  call). 

3.  _ Conduct  communication  check  with  each  SNAP  and  DIS  logger  operator. 

4.  _ Boot  all  equipment. 

5.  _ Verify  with  NETSNOOP  that  each  logger  is  connected  to  the  network. 

6.  _ Verify  that  NTP  is  running. 

7.  _ Synchronize  the  display  clock  to  the  NTP  derived  clock. 

8.  _ Start  NETLOOK  and  verify  operation. 

9.  _ Verify  that  all  machines  at  the  test  control  site  are  up. 

10.  _ Verify  that  the  DIS  logger  at  each  site  is  up  and  ready  to  receive  PDUs. 

1 1 .  _ Verify  that  the  DIS  logger  at  each  site  is  set  to  port  6996  and  the  exercise  ID  is  0. 

12.  _ Direct  the  SIMLAB  to  replay  a  log  file  for  remote  machines  to  use  for  system 

checks.  (May  not  be  received  by  the  WSSF) 

13.  _ Verify  that  each  site  is  receiving  the  log  file. 

14.  _ Using  NETSNOOP  verify  that  the  appropriate  network  traffic  b  present  on  each 

network  segment. 

15.  _ Direct  SIMLAB  to  stop  playback. 


E2-1 


LINKED  SIMULATORS  PHASE 


NETWORK  COORDINATOR  CHECKLIST 

1 6.  _ Verify  with  the  DIS  logger  operators  at  each  site  that  all  log  files  have  been 

created  and  are  empty.  The  format  is: mmddyy_t es t #_lab .  lgr  (Is  - 

al  /disk2/stricom/log/) 

17.  _ _  Report  the  results  of  the  system  checks  to  the  Test  Controller. 

18.  _ Leave  the  voice  conference  net. 

1 9.  _ Attend  mission  brief. 

20.  _ Supervise  repairs  as  necessary  to  prepare  equipment  for  the  next  day  (T-3  hrs.  to 

T-60  min.). 
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LINKED  SIMULATORS  PHASE 


NETWORK  COORDINATOR  CHECKLIST 

DAY  OF  TEST  BEFORE  TEST  RUNS 

Network  Coordinator: _ 

Date: _  Lab  Time: _ to _ 

In  the  spaces  provided,  record  the  time  each  action  is  completed. 

1 .  _  Arrive  on  site. 

2.  _  Open  written  logs. 

3.  _  Check  supplies  (clipboards,  checklists,  forms,  file  name  lists,  pens  &  pencils,  scratch 

paper,  backup  media). 

4.  _  Join  the  support  voice  conference  net  by  dialing  (619)  939-6338  (SIMLAB  operator  wll 

make  the  first  call). 

5.  _  Conduct  communication  check  with  all  participants. 

6.  _  Boot  all  equipment. 

7.  _  Verify  equipment  connections  (Ethernet,  keyboard,  mouse). 

8.  _  Verify  NTP  is  running  and  synchronized  to  the  display  clock. 

9.  _  Start  NETLOOK  and  verify  operation. 

10.  _  Verify  all  machines  are  up. 

1 1 .  _  Verify  that  the  DIS  logger  at  each  site  is  up  and  ready  to  receive  PDUs. 

12.  _  Verify  that  the  DIS  logger  at  each  site  isset  to  port  6996  and  the  exercise  ID  is  0. 

13.  _  Direct  the  SIMLAB  to  replay  a  log  file  for  remote  machines  to  use  for  system  checks. 

14.  _  Verify  each  site  is  receiving  the  log  file. 

1 5.  _  Verify  network  traffic  with  each  site. 

16.  _  Direct  SIMLAB  to  stop  playback. 

17.  _  Verify  with  each  site  that  all  log  files  have  been  created  and  are  empty.  The  format  is: 

mmddyy_test#_lab . lgr  (Is  -al  /disk2 /stricom/log/) 
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LINKED  SIMULATORS  PHASE 


NETWORK  COORDINATOR  CHECKLIST 

FOR  EACH  TEST  RUN 

Network  Coordinator: _ 

Date: _  Lab  Time: _ to _ 

1.  Use  data  sheets  with  filenames. 

2.  When  told  by  the  Test  Controller,  direct  all  operators  to  start  recording  data 
(“Recorders  ON”). 

3.  Test  run.  Record  data. 

4.  When  told  by  the  Test  Controller,  direct  all  operators  to  stop  recording  data 
(“Recorders  OFF”). 

5.  Direct  all  operators  to  close  test  file. 

6.  Confirm  that  each  site  has  logged  required  data  in  the  written  log. 

7.  Direct  all  SNAP  operators  to  open  the  next  logger  file. 

During  break 

7.  Direct  operators  to  run  system  checks,  if  necessary. 

8.  Direct  operators  to  run  ping  tests,  if  necessary. 

9.  Direct  operators  to  run  latency  tests,  if  necessary. 

10.  Direct  operators  to  run  time  synchronization  tests,  if  necessary. 

1 1 .  Direct  SNAP  operators  to  compress  their  data. 

12.  Direct  F-14  WSIC  to  change  VCR  tapes. 

13.  Report  completion  and  results  to  the  Test  Controller. 


LINKED  SIMULATORS  PHASE 


NETWORK  COORDINATOR  CHECKLIST 

END  OF  DAY 

Network  Coordinator:  _ _ _ 

Date: _ _  Lab  Time: _ to  _ _ 

In  the  spaces  provided,  record  the  time  each  action  is  completed. 

1.  _ Ensure  data  is  in  the  proper  directory  with  the  correct  file  name.  Include:  logger 

files,  SNAP  files,  ping  files,  time  synch  files,  and  system  check  files.  Format  for 
directory  is:  disk2  /data/mmddyy/machine_name/ 

2.  _ RCP  F-14  WSIC  files  to  the  BMIC. 

3.  _ Ensure  F-18  WSSF  files  are  at  the  SIMLAB. 

4.  _ RCP  SIMLAB  files  to  the  BMIC. 

5.  _ Backup  data  to  tape  using  the  tar  -cvf  command  (or  tape  tool). 

6.  _ Verify  backup  using  th Qtar  -tf  command  (or  tape  tool). 

7.  _ Label  the  tape. 

8.  _ Compress  the  files  on  disk  in  the/disk2  /data/nunddyy /machine_name/ 

directory. 

9.  _ Leave  support  voice  conference  net. 

10.  _ Turn  off  all  equipment. 

1 1 .  _ Participate  in  mission  debrief. 
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LINKED  SIMULATORS  PHASE 


SITE  SUPERVISOR  CHECKLIST 

DAY  BEFORE  TEST 

Site:  (circle  one)  F-14  WSIC  F-l 8  WSSF  SIMLAB 


Supervisor:  _ _ _ 

Date: _  Lab  Time: _ to _ 

In  the  spaces  provided,  record  the  time  each  action  is  completed. 

1.  _ Join  the  command  voice  conference  net  by  dialing  (619)  939-0010  (SIMLAB 

supervisor  will  make  the  first  call) 

2.  _ Receive  reports  from  the  SNAP  and  DIS  logger  operators  at  your  site  concerning 

the  status  of  their  equipment. 

3 .  ■ _ Report  site  status  to  the  Test  Controller. 

4.  _ F-14  WSIC  and  F-18  WSSF  Site  Supervisors  leave  voice  conference  net. 

SIMLAB  Site  Supervisor  end  voice  conference  net  when  all  other  participants 
have  signed  off. 

5.  _ Brief  mission. 

A.  _  Discuss  equipment  readiness  and  any  repairs  required. 

B.  _  Establish  arrival  times  of  key  personnel. 

C.  _  Review  mission  procedures  and  test  cards. 

D.  _  Ensure  that  your  logger  operators  have  enough  supplies  for  the 

entire  test  period  on  hand  (checklists,  file  lists,  blank 
hardware/software  discrepancy  log  sheets,  pens  or  pencils,  scratch 
paper,  4mm  tape  cartridges,  crypto  keys,  safe  combinations). 

6.  _ Coordinate  repairs  as  necessary  to  prepare  equipment  for  the  next  day  (T-23hrs. 

to  T-60  min.). 
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LINKED  SIMULATORS  PHASE 


SITE  SUPERVISOR  CHECKLIST 
DAY  OF  TEST  BEFORE  TEST  RUNS 

Site:  (circle  one)  F-14  WSIC  F-18  WSSF  SIMLAB 


Supervisor: _ _ 

Date: _  Lab  Time: _ to _ 

In  the  spaces  provided,  record  the  time  each  action  is  completed. 

1.  _ Join  the  command  voice  conference  net  by  dialing  (619)  939-0010  (SIMLAB 

supervisor  will  make  the  first  call). 

2.  _ Ensure  logger  operators  have  the  applicable  checklists,  blank  log  sheets,  file  lists, 

pens  or  pencils,  scratch  paper  and  backup  media). 

3.  _ Receive  reports  from  the  SNAP  and  STRICOM  logger  operators  at  your  site 

concerning  the  status  of  their  equipment. 

4.  _ Verify  with  the  laboratory  supervisor  that  the  simulation  is  fully  functional. 

Cockpit  switchology  (Card  A)  (WSSF) 

IN S/GP  S/S AHRS/TMS  (WSIC) 

Gas  bottles  (SIMLAB) 

5.  _ Label  all  removaHe  storage  media. 

6.  _ Report  site  status  to  the  Test  Controller. 
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LINKED  SIMULATORS  PHASE 


STTE  SUPERVISOR  CHECKLIST 

FOR  EACH  TEST  RUN 

Site:  (circle  one)  F-14  WSIC  F-18  WSSF  SIMLAB 

Supervisor: _ 

Date: _  Lab  Time: _ to _ 

1.  Ensure  simulation  recorders  are  running. 

2.  Report  site  ready  for  next  run. 

3.  When  directed  by  the  Test  Controller  direct  simulation  operators  to  start  simulations. 

4.  F-18  WSSF  back  up  pilot  on  “Fox”  call. 

5.  F-18  WSSF  record  data  from  display  for  quick  look.  Report  completion  to  Test  Controller. 

6.  SIMLAB  verify  shot  box  not  exceeded. 
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LINKED  SIMULATORS  PHASE 


SITE  SUPERVISOR  CHECKLIST 
END  OF  DAY 

Site:  (circle  one)  F-14  WSIC  F-18  WSSF  SIMLAB 

Supervisor: _ 

Date: _  Lab  Time: _ to _ 

In  the  spaces  provided,  record  the  time  each  action  is  completed. 

1.  _ Gather  checklists,  logs,  notes  and  removable  storage  media. 

2.  _ SIMLAB  print  log  file. 

3.  _ Recheck  labeling  on  logs  and  removable  media. 

4.  _ Conduct  site  debrief. 

5.  _ Leave  voice  conference  net. 

6.  _ Participate  in  mission  debrief  with  all  sites. 
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LINKED  SIMULATORS  PHASE 


SNAP  OPERATOR  CHECKLIST 

DAY  BEFORE  TEST 

Site:  (circle  one)  F-14  WSIC  F-18  WSSF  SIMLAB 


Operator: _ 

Date: _  Lab  Time: _ to _ 

In  the  spaces  provided,  record  the  time  each  action  is  completed. 

1 .  _ Ensure  enough  supplies  for  the  entire  test  period  are  on  hand  (checklist,  blank 

hardware/software  discrepancy  log  sheets,  file  list,  pens  or  pencils,  scratch  paper, 
4mm  tape  cartridges). 

2.  _ Verify  equipment  connections  (Ethernet  #1,  Ethernet  #2,  IRIG-B,  keyboard, 

mouse).  (Cannot  be  done  at  the  WSSF) 

3.  _ Join  the  support  voice  conference  net  by  dialing  (619)  939-6338  (SIMLAB 

operator  will  make  the  first  call). 

4.  _ Start  SNAP  computer  per  the  attached  procedures. 

5.  _ Set  SNAP  to  filter  on  appropriate  PDU  and  Ethernet  data. 

6.  _ Test  SNAP  operation  by  observing  pings  sent  by  the  DIS  logger  op<rator  at  your 

site.  (Cannot  be  done  at  the  WSSF) 

7.  _ Report  to  the  Network  Coordinator  when  you  are  ready  to  conduct  the  data 

collection  test. 

8.  _ When  directed  by  the  Network  Coordinator  conduct  a  data  collection  test  by 

receiving  specified  data.  (Cannot  be  done  at  the  WSSF) 

9.  _ Report  the  results  of  system  checks  to  your  Site  Supervisor  and  the  Network 

Coordinator. 

10.  _ Leave  the  voice  conference  net. 

1 1 .  _ Attend  mission  brief. 

12. _ _ Make  repairs  as  necessary  (T-23  hrs.  to  T-60  min.). 
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LINKED  SIMULATORS  PHASE 
SNAP  OPERATOR  CHECKLIST 
DAY  OF  TEST  BEFORE  TEST  RUNS 

Site:  (circle  one)  F-14  WSIC  F- 1 8  WSSF  SIMLAB 
Operator:  _ _ _ 

Date: _  Lab  Time: _ to _ 

In  the  spaces  provided,  record  the  time  each  action  is  completed. 

1.  _ Arrive  on  site. 

2.  _ Open  written  log. 

3.  _ Ensure  enough  suppliesfor  the  entire  test  period  are  on  hand  (checklist,  blank 

hardware/software  discrepancy  log  sheets,  file  list,  pens  or  pencils,  scratch  paper, 
4mm  tape  cartridge). 

4.  _ Verify  equipment  connections  (Ethernet  #1,  Ethernet  #2,  IRIG-B,  keyboard, 

mouse). 

5.  _ Join  the  support  voice  conference  net  by  dialing  (619)  939-6338  (SIMLAB 

operator  will  make  the  first  call). 

6.  _ Start  SNAP  computer  per  the  attached  procedures. 

7.  _ Set  SNAP  to  filter  on  appropriate  PDU  and  Ethernet  data. 

8.  _ Test  SNAP  operation  by  observing  pings  sent  by  the  DIS  Logger  operator  at  your 

site. 

9.  _ Report  to  the  Network  Coordinator  when  you  are  ready  to  conduct  the  data 

collection  test. 

10.  _ When  directed  by  the  Network  Coordinator,  conduct  a  data  collection  test  by 

sending  and  receiving  specified  data. 

11.  _ Report  the  results  of  system  checks  to  your  Site  Supervisor  and  the  Network 

Coordinator. 

12.  _ Run  “(DA TE) files. bat'  to  create  files.  Verify  all  files  have  been  created  and  are 

empty. 
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LINKED  SIMULATORS  PHASE 


SNAP  OPERATOR  CHECKLIST 

FOR  EACH  TEST  RUN 

Site:  (circle  one)  F-14  WSIC  F-18WSSF  SIMLAB 


Operator: _ 

Date: _  Lab  Time: _ to _ 

1.  Start  recording  when  directed  by  the  Network  Coordinator.  Record  PDUs  and  Ethernet 
packets  from  all  installed  Ethernet  cards.  Verify  that  PDUs  and  Ethernet  packets  are  on  the 
network. 

2.  Stop  recording  when  directed  by  the  Network  Coordinator,  NOT  BEFORE. 

3.  Save  data  to  file. 

4.  Check  file  name  off  run  sheet. 

5.  Report  to  your  Site  Supervisor  and  the  Network  Coordinator  when  you  are  ready  for  the  next 
run. 

During  break 

6.  Compress  files. 

7.  Report  completion  to  your  Site  Supervisor  and  to  the  Network  Coordinator. 
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LINKED  SIMULATORS  PHASE 


SNAP  OPERATOR  CHECKLIST 

END  OF  DAY 

Site:  (circle  one)  F-14  WSIC  F-18  WSSF  SIMLAB 

Operator: _ 

Date:  _ _  Lab  Time: _ to _ 

In  the  spaces  provided,  record  the  time  each  action  is  completed. 

1 .  _ _ Ensure  the  data  is  in  the  proper  directory  (c :  /  SNAP)  with  the  proper  filename. 

2.  _ Reboot  SNAP  into  Windows  95  mode. 

3 .  _ Backup  data  files  into  a  zip  file  using  WIN  Zip. 

4.  _ F-14  WSIC  and  F-18  WSSF  RCP  files  to  the  BMIC  and  SIMLAB  respectively. 

5.  _ Quit  Windows  95  and  shut  down  computer. 

6.  _ Leave  voice  conference  net. 

7.  _ Give  all  checklists,  written  logs,  notes,  file  lists  and  removable  media  to  your  Site 

Supervisor. 

8.  _ Attend  mission  debrief. 
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LINKED  SIMULATORS  PHASE 


SNAP  OPERATOR  CHECKLIST 

SNAP  START-UP  PROCEDURES 

Site:  (circle  one)  F-14  WSIC  F-18  WSSF  SIMLAB 
Operator: _ _ 

Date:  _ _  Lab  Time: _ to _ 

1.  Turn  on  power 

A.  Wait  for  the  system  to  boot  to  the  startup  menu 

B.  Select  option  2  from  the  menu 

C.  Press  Alt-SysReq 

D.  Login  —  snap 

E.  Password  -  snap 

F.  snap 

G.  Press  Alt-SysReq 

H.  Type  cd  snap 

I.  Press  Alt-SysReq 

J.  Type  snap 

2.  Initial  Setup 

A.  Choose  FILE  from  menu,  then  LOAD  CONFIGURATION 

B.  Select  appropriate  configuration  file 

C.  Click  on  RETURN 

3.  Configure 

A.  Confirm  GPS  is  set  to  YES 

B.  Select  ETHERNET  #1  (F-14  WSIC)  or  BOTH  (F-18  WSSF  and  SIMLAB) 

C.  Click  SAVE  I/O  SCHEDULE 

4.  I/O  Setup 

A.  Ethernet  #1  receive  setup 

1.  BROADCAST 

2.  Set  for  data  to  collect  (PDU  Headers  Only) 

3.  Click  SAVE  ETHERNET  SETUP 

B.  Ethernet  #2  receive  setup 

1.  PROMISCUOUS 

2.  Set  for  data  to  collect  (ETHERNET  HEADERS  and  VARIABLE  DATA:  150 

bytes). 

3.  Click  SAVE  ETHERNET  SETUP 

C.  Click  on  RETURN 
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LINKED  SIMULATORS  PHASE 
SNAP  OPERATOR  CHECKLIST 


5.  OPERATE 

A.  DISPLAY  MODE 

B.  Enter  screen  for  PDU  GRAPH  (display  Ethernet  #1  OR  #2)  or  Statistics  (display  Ethernet 
#1  AND  #2) 

C.  Press  START  to  record  and  view 

D.  Press  STOP  to  stop  recording 

E.  Press  SAVE  DATA 

F.  Give  file  the  next  name  from  the  file  list 
*  Repeat  steps  5A  through  5F  for  each  run 


TO  COMPRESS  FILES 

1 .  Click  on  QUI T  from  SNAP  main  screen 

2.  Press  RESET  Button 

3 .  Enter  Windows  95  (menu  option  #  1 ) 

4.  Type  snap  at  password  screen 

5.  Go  into  Windows  Explorer 

—  Data  files  are  stored  in  c :  /  snap 

6.  Highlight  the  files  to  be  zipped 

7 .  On  highlighted  files,  click  right  mouse  button 

8.  Choose  ADD  TO  ZIP 

9.  Choose  I  AGREE 

10.  Enter  file  name  (name.zip) 

1 1 .  Exit  Windows  95  to  SNAP 

A.  Go  to  START  (bottom  of  screen  tool  bar,  lights  up  when  mouse  arrow  points  to  area) 

B.  Choose  SHUT  DOWN 

C.  Choose  RESTART  COMPUTER 

12.  Return  to  step  1 A  on  previous  page  to  record  more  data. 

TO  FTP  ZIP  FILES  TO  DIS  LOGGER  COMPUTER  FOR  SAVING  AND  TAPING 

1 .  Choose  WinFTP  in  Windows  95,  then  CONNECT  (at  bottom  of  screen) 

2.  Type  in  destination  computer  name  (IP#)  and  host  ID(ie  tcac_indy ) 

3 .  For  your  ID  type  disl  og 

4.  Type  in  password  (same  password  as  used  to  enter  the  DIS  logger) 

5.  Choose  OK 

6.  When  you  have  verified  the  connection,  choosedi  sk2  in  remote  computer  files  screen 

A.  choose  the  file  name  corresponding  to  your  location 

B.  open  this  file 

7.  Highlight  the  .  ZIP  file  to  be  transferred 
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LINKED  SIMULATORS  PHASE 


SNAP  OPERATOR  CHECKLIST 

8 .  Click  on  the  right  arrow  button  in  the  center  between  the  directories 

9.  Verify  that  the  files  have  been  transferred 

TO  SET  UP  PRE-LABELED  FILES  IN  SNAP  BEFORE  THE  TEST 

1 .  Run  batch  file,  “(date)files.bat”,  from  c :  / snap  directory  (60  blank  files  will  be 
created) 

2.  This  will  create  “Required  Parameter  Missing”  errors;  IGNORE  THESE  MESSAGES 

3.  To  edit  and  change  date 

A.  Type  edit  (date)files.bat 

B.  Press  Alt  S  (for  search  menu) 

C.  Select  CHANGE 

D.  Change  data  from  0801r  to  0805r,  or  other  appropriate  date 

E.  Select  CHANGE  ALL 

F.  Select  FILE  then  SAVE  to  save  these  changes 

G.  Select  FILE  then  EXIT  to  exit  back  to  c :  /  snap 
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LINKED  SIMULATORS  PHASE 


PIS  LOGGER  OPERATOR  CHECKLIST 

DAY  BEFORE  TEST 

Site:  (circle  one)  F- 14  WSIC  F-18WSSF  SIMLAB  BMIC/TCAC 

Operator:  _ _ 

Date: _  Lab  Time: _ to _ 

In  the  spaces  provided,  record  the  time  each  action  is  completed. 

1 .  _ Ensure  enough  supplies  for  the  entire  test  period  are  on  hand  (checklist,  blank 

hardware/software  discrepancy  log  sheets,  pens  or  pencils,  scratch  paper,  4mm 
tape  cartridges). 

2.  _ Join  the  support  voice  conference  net  by  dialing  (619)  939-6338  (SIMLAB 

operator  will  make  the  first  call). 

3.  _ Boot  all  equipment. 

4.  _ Turn  off  screen  savers. 

5.  _ Start  STRICOM  logger. 

6.  _ Open  test_setup  file  (create,  if  necessary).  Change  port  number  to  6996  and 

exercise  ID  to  0,  then  apply  the  changes. 

7.  _ Verify  all  log  files  have  been  created  and  are  empty.  The  format  is: 

mmddyy_test#_lab. lgr  (Is  -al  /disk2 /stricom/log/) 

8.  _ Coordinate  with  the  SNAP  operator  at  your  site  to  test  SNAP.  (Cannot  be  done  at  the 

WSSF) 

9.  _ Conduct  system  test.  Select  “system  test”  from  the  menu  in  the  upper  left  comer 

of  your  screen.  Verify  the  data  in /disk2  /  system/  system_check .  log 

10.  _ Conduct  time  synchronization  test.  Select  “time  synch  test”  from  the  menu  in  the 

upper  left  comer  of  your  screen.  Record  and  report  to  the  Network  Coordinator 
time  differences  of  0.5  milliseconds  or  greater  .(Cannot  be  done  at  the  WSSF) 

1 1 .  _ Conduct  ping  test.  Select  “ping  test”  from  the  menu  in  the  upper  left  comer  of 

your  screen.  Verify  data  in/disk2/ping/ping_test .  log  file.  (Cannot  be 
done  at  the  WSSF) 
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PIS  LOGGER  OPERATOR  CHECKLIST 


12.  _ Report  to  the  Network  Coordinator  when  you  are  ready  to  conduct  the  data 

collection  test. 

13.  _ When  directed  by  the  Network  Coordinator,  conduct  a  data  collection  test  by 

sending  and  receiving  specified  data.  The  SIMLAB  operator  will  send  a  log  file  for 
the  SNAP  and  other  DIS  logger  operators  to  receive. 

14.  _ Verify  logger  sees  DIS  entities  by  using  tho“ statistics”  push  button. 

15.  _ Record  tes t_setup  logger  file. 

16.  _ Stop  logger  at  the  end  of  the  test. 

17.  _ Run  “count _pdus ”  (using  test_setup .  lgr  as  input  file).  Report  problems  to 

the  Network  Coordinator. 

1 8.  _ Report  the  results  of  system  checks  to  your  Site  Supervisor  and  the  Network 

Coordinator. 

19.  _ Leave  the  voice  conference  net. 

20.  _ Attend  mission  brief. 

21.  _ Make  repairs  as  necessary  (T-23hrs.  to  T-60  min.). 
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T  JNKEP  SIMULATORS  PHASE 
PIS  LOGGER  OPERATOR  CHECKLIST 
DAY  OF  TEST  BEFORE  TEST  RUNS 

Site:  (circle  one)  F-14  WSIC  F-18  WSSF  SIMLAB 
Operator:  _ _ 

Date: _  Lab  Time: _ _  to _ : _ 

In  the  spaces  provided,  record  the  time  each  action  is  completed. 

1.  _ Arrive  on  site. 

2.  _ Open  written  log. 

3 .  _ _ Ensure  enough  supplies  for  the  entire  test  period  are  on  hand  (checklist,  blank 

hardware/software  discrepancy  log  sheets,  pensorpencils,  scratch  paper,  4mm  tape 
cartridges). 

4.  _ Join  the  support  voice  conference  net  by  dialing  (619)  939-6338  (SIMLAB 

operator  will  make  the  first  call). 

5.  _ Boot  all  equipment. 

6.  _ Turn  off  screen  savers. 

7.  _ Start  STRICOM  logger. 

8.  _ Open  test_setup  file  (create,  if  necessary).  Change  port  number  to  6996  and 

exercise  ID  to  0,  then  apply  changes. 

9.  _ Verify  all  log  files  have  been  created  and  are  empty.  The  format  is: 

nimddyy_test#_lab.  lgr  (Is  -al  /disk2/stricom/log/) 

10.  _ Coordinate  with  the  SNAP  operator  at  your  site  to  test  SNAP. 

11.  _ Conduct  System  Test.  Select  “ system  test ”  from  the  menu  in  the  upper  left  comer 

of  your  screen,  verify  data  in/disk2  /system/  system_check .  log 

12.  _ Conduct  time  synchronization  test.  Select  “ time  synch  test”  from  the  menu  in  the 

upper  left  comer  of  your  screen.  Record  and  report  time  differences  of  0.5 
milliseconds  or  greater. 
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PIS  LOGGER  OPERATOR  CHECKLIST 


13.  _ Conduct  ping  test.  Select  “ping  test ”  from  the  menu  in  the  upper  left  comer  of 

your  screen.  Verify  data  in  /disk2  /ping/ping_test .  log  file. 

14.  _ _  Report  to  the  Network  Coordinator  when  you  are  ready  to  conduct  the  dat  a 

collection  test. 

15.  _ When  directed  by  the  Network  Coordinator,  conduct  a  data  collection  test  by 

sending  and  receiving  specified  data.  The  SIMLAB  operator  will  send  a  log  file  for 
the  SNAP  and  other  DIS  logger  operators  to  receive. 

16.  _ Verify  logger  sees  DIS  entities  by  using  ^“statistics  ”  push  button. 

17.  _ Record  test_setup  logger  file. 

1 8.  _ Stop  logger  at  the  end  of  the  test. 

19.  _ Run  “ count _pdus ”  (using  test_setup .  lgr  as  input  file).  Report  problems  to 

the  Network  Coordinator. 

20.  _ Report  the  results  of  system  checks  to  your  Site  Supervisor  and  the  Network 

Coordinator. 

22. _ Ensure  the  first  test  logger  file  is  open  and  ready  to  record. 
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PIS  LOGGER  OPERATOR  CHECKLIST 
FOR  EACH  TEST  RUN 

Site;  (circle  one)  F-14  WSIC  F-18WSSF  SIMLAB 
Operator: _ _ 

Date: _  Lab  Time: _ to _ 

1 .  Use  data  sheets  with  filenames. 

2.  Record  data  when  directed  by  the  Network  Coordinator. 

3.  Test  run.  Record  data. 

4.  Stop  recording  data  when  directed  by,  and  NOT  BEFORE,  the  Network  Coordinator. 

5.  Close  test  file. 

6.  Report  to  your  Site  Supervisor  and  the  Network  Coordinator  whether  data  collection  was 
successful. 

7.  Open  the  next  logger  file. 

During  Break 

8.  Run  system  check. 

9.  Run  ping  test. 

10.  Run  latency  test. 

1 1 .  F- 14  WSIC  change  VCR  tapes. 

12.  Report  completion  to  your  Site  Supervisor  and  the  Network  Coordinator. 


T  INKED  SIMULATORS  PHASE 
PIS  LOGGER  OPERATOR  CHECKLIST 
END  OF  DAY 

Site:  (circle  one)  F- 1 4  WSIC  F- 1 8  WSSF  SIMLAB 
Operator:  _ _ _ 

Date: _  Lab  Time: _ to _ 

In  the  spaces  provided,  record  the  time  each  action  is  completed. 

1 .  _ Ensure  data  is  in  the  proper  directory  with  the  correct  file  name.  Include:  logger 

files,  SNAP  files,  ping  files,  time  synch  files,  and  system  check  files.  Format  for 
directory  is:  /disk2/data/mmddyy/machine_name/  (e.g. 

/disk2 /data/ 082 69 6 /wssf -dislog/) 

2.  _ Backup  all  logger  files  to  a  single  tar  disk  file  (nmddyy_s i t ename .  lgr .  tar) 

using  the  tar  -cvf  command.  Compress  the  file  using  th ecompress  utility. 

3.  _ SIMLAB  and  BMIC  backup  data  to  tar  tape  file  using  the  tar  -cvf  command  (or 

tape  tool). 

4.  _ Verify  backup  using  the  tar  -tf  command  (or  tape  tool). 

5.  _ Compress  the  disk  files  in  the/disk2 /data/mmddyy/machine_name/ 

directory. 

6.  _ Delete  the  .  lgr  files. 

7.  _ Leave  voice  conference  net. 

8.  _ Turn  off  all  equipment. 

9.  _ Give  all  checklists,  written  logs,  notes,  4mmtape  cartridge  and  VCR  tapes  (if 

applicable)  to  the  Site  Supervisor. 

1 0.  _ Attend  mission  debrief. 
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TEST  CARDS 
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JADS  LSP 

GENERAL 

MTfsSTON: 

DATE: 

OPNO: 

TIME: 

WSSF 

PILOT: 

SIMLAB 

OPERATOR: 

WSIC 

PILOT: 

TCAC 

CONTROLLER: 

RIO: 

WSSF  CONFIGURATION 

F/A-18C/OFP  11C 

ATM-QM  STATION 

SIMLAB  CONFIGURATION 

AIM-9MR  (AIM-9M(8/9)) 

SF.F.KF.R  SERIAL  #: 

WSIC  CONFIGURATION 

F-14D/OFP  D03.2 

NOTES 

COVER  CARD 


LINKED  SIMULATORS  PHASE 


_ F-18  WSSF  PROCEDURES _ 

BEFORE  FIRST  RUN 

1.  INS-NAV 

2.  RADAR  -  OPERATE 

3.  SMS  POWER -ON 

4.  FLAPS -AUTO 

5.  PING  EACH  REMOTE  NODE 

BEFORE  EACH  RUN 

1.  LOAD  NEW  MISSILE 

A.  ENTER  NAV  MASTER  MODE  (DESELECT  A/A  MASTER  MODE) 

B.  LANDING  GEAR  POSITION  -  DOWN 

C.  LEFT  WEIGHT  ON  WHEELS  -  ON 

D.  RIGHT  WEIGHT  ON  WHEELS  -  ON 

E.  PARKING  BRAKE  -  ON 

2.  RESET  SIMULATION 

3.  CHECK  MNS-1  SMS  LINK 

4.  SELECT  DIRECT  CONTROL  AIRFRAME  MODEL 
INITIAL  CONDITIONS:  12,000'  MSL;  450  KTAS;  3.0  AOA 

5.  INITIALIZE  VAX  AND  HP-735 

6.  PARKING  BRAKE  -  OFF 

7.  WHEN  AIRBORNE 

A.  LEFT  WEIGHT  ON  WHEELS  -  OFF 

B.  RIGHT  WEIGHT  ON  WHEELS  -  OFF 

C.  LANDING  GEAR  POSITION  -  UP 

D.  A/A  MASTER  MODE  SELECT  -  ON 

E.  MASTER  ARM  -  ARM 

8.  SIMULATION  DATA  RECORDING  -  START 


CARD  A 
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_ F-18  WSSF  PROCEDURES _ 

TO  START  EACH  RUN 

1 .  SIMULTANEOUSLY  SELECT  KOHLMAN  AIRFRAME  MODEL  AND 
PROFILE  RECORDING  -  ON 

2.  SMS  RECORDER  -  ON 

DURING  EACH  RUN 

1 .  A/A  WEAPON  SELECT  -  SIDEWINDER 

2.  SENSOR  CONTROL  -  AUTO  ACQ 

3.  CAGE/UNCAGE- UNCAGE 

4.  TRIGGER  -  PULL  TO  SECOND  DETENT 

AFTER  EACH  RUN 

1 .  SIMULATION  DATA  RECORDING  -  OFF 

2.  REPORT  QUICK  LOOK  DATA 

3 .  SAVE  RECORDED  FLIGHT  PROFILE  FOR  PLAYBACK 

4.  MANUALLY  RECORD  QUICK  LOOK  DATA 

5.  ARCHIVE  SAVED  FLIGHT  PROFILE 


CARD  A 
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JADS  LSP 


INITIALIZE:  _ 

- ►  WSIC  (  Tgt)  -  End 

-  WSIC  (  Tgt)  -  Start 

10.4  KFT  /  0.72  imn  / 1 80  deg  TAA 


i -  WSSF  (Shooter) 

11.3  KFT/ 0.71  imn /in  trail 


A.  INITIALIZE  WSSF,  WSIC,  AND  SIMLAB 

B.  VERIFY  BASELINE  SYSTEM  SWITCHOLOGY 

1.  (EACH  LAB)  “LAB  READY” 

2.  (TCAC)  VERIFY  WITH  NETWORK  COORDINATOR  THAT  THE  DATA 


3.  (TCAC) 

4.  (TCAC) 

5.  (TCAC) 

6.  (TCAC) 

7.  (TCAC) 

8.  (TCAC) 

9.  (TCAC) 


LOGGERS  ARE  READY. 


“PROFILE  # _ ;  PASS  # _ .” 

DIRECT  NETWORK  COORDINATOR  TO  START  RECORDERS 
**NOTE  TIME**  “RECORDERS  ON” 

NETWORK  COORDINATOR  REPORTS  RECORDERS  RUNNING 
“3-2-1-START  RUN” 

“Step  3” 

“KNOCK  IT  OFF” 

“RECORDERS  OFF” 


10.  (TCAC)  “RECORDERS  OFF” 

1 1 .  (F-l 8)  “DATA  CHECK  COMPLETE” 

12.  (SIMLAB)  “DATA  CHECK  COMPLETE” 

1 3 .  (TCAC)  “RESET  SIMULATORS” 

14.  (EACH  LAB)  “SIMULATORS  RESET” 

15.  (EACH  LAB)  TO  CONTINUE,  GO  TO  STEP  3. 
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JADS  LSP 

INITIALIZE: 

_  NM 

- ►  WSIC  (  Tgt)  -  End 

-  WSIC  (  Tgt)  -  Start 

4— -  WSSF  (Shooter) 

10.4  KFT  /  0.72  imn  / 180  deg  TAA 

11.3  KFT/ 0.71  imn /in  trail 

A.  INITIALIZE  WSSF,  WSIC,  AND  SIMLAB 

B.  VERIFY  BASELINE  SYSTEM  SWITCHOLOGY 

1.  (EACH  LAB)  “LAB  READY” 

2.  (TCAC)  VERIFY  WITH  NETWORK  COORDINATOR  THAT  THE  DATA 

LOGGERS  ARE  READY. 

3.  (TCAC)  “PROFILE  # _ ;  PASS  # _ 

4.  (TCAC)  “STEP  1” 

5.  (SIMLAB)  “15  SECONDS  TO  GO” 

6.  (TCAC)  DIRECT  NETWORK  COORDINATOR  TO  START  RECORDERS 

7.  (TCAC)  **NOTE  TIME**  “RECORDERS  ON” 

8.  (TCAC)  NETWORK  COORDINATOR  REPORTS  RECORDERS  RUNNING 

9.  (TCAC)  “3-2-1-START  RUN” 

10.  (SIMLAB)  “STEP  2” 

11.  (TCAC)  “STEP  3” 

12.  (TCAC)  “KNOCK  IT  OFF” 

13.  (TCAC)  **NOTE  TIME**  “RECORDERS  OFF” 

14.  (F-l  8)  “DATA  CHECK  COMPLETE” 

15.  (SIMLAB)  “DATA  CHECK  COMPLETE” 

16.  (TCAC)  “RESET  SIMULATORS” 

17.  (EACH  LAB)  “SIMULATORS  RESET” 

18.  (EACH  LAB)  TO  CONTINUE,  GO  TO  STEP  3. 


PASS#:  _  PROFILE#:  V2 
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O 

£ 

I- 

z 

O 

o 

H 

(/) 

LU 


MISSION: _  DATE: 
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MISSION: _  DATE: 
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ACRONYM  LIST 
for  the 

SYSTEM  INTEGRATION  TEST 
LINKED  SIMULATORS  PHASE 


ACRONYM  LIST 


6-DOF 

ADI 

ADS 

AMRAAM 

ARPA 

BMIC 

C4I 

CCM 

CCR 

CM 

COI 

COMSEC 

CPU 

csu 

DEC 

DIS 

DDT&E 

DMAP 

DMSO 

DSI 

DSU 

DT 

DT&E 

ES 

ESSM 

ETE 

GCS 

GMT 

GPS 

HUD 

H/W 

HWIL 

ICD 

IR 


Six  (6)  Degree-of-Freedom 

Applied  Dynamics  International 
Advanced  Distributed  Simulation 
Advanced  Medium  Range  Air-to-Air  Missile 
Advanced  Research  Projects  Agency 

Battle  Management  Interoperability  Center 

Command,  Control,  Communications,  Computers,  and  Intelligence 

Counter-Countermeasures 

Captive  Carry  Reliability 

Countermeasures 

Critical  Operational  Issue 

Communications  Security 

Central  Processor  Unit 

Channel  Service  Unit 

Digital  Equipment  Corporation 
Distributed  Interactive  Simulation 

Deputy  Director,  Test,  Systems  Engineering  and  Evaluation  (Test  and 
Evaluation) 

Data  Management  and  Analysis  Plan 

Defense  Modeling  and  Simulation  Organization 

Defense  Simulations  Internet 

Data  Service  Unit 

Developmental  Testing 

Developmental  Test  and  Evaluation 

Entity  State 

Evolved  Sea  Sparrow  Missile 
End-to-End  Test 

Guidance  and  Control  Section 
Greenwich  Mean  Time 
Global  Positioning  System 

Heads-Up  Display 
Hardware 

Hardware-in-the-Loop 

Interface  Control  Document 
Infrared 
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IRCM 

Infrared  Countermeasures 

IRCCM 

Infrared  Counter-Countermeasures 

IRIG 

Interrange  Instrumentation  Group 

ITP 

Integration  Test  Plan 

JADS 

Joint  Advanced  Distributed  Simulation 

JDAM 

Joint  Direct  Attack  Munition 

JIOT&E 

Joint  Initial  OT&E 

JORD 

Joint  Operational  Requirements  Document 

JSPO 

Joint  System  Program  Office 

JT&E 

JADS  Joint  Test  and  Evaluation 

JTD 

Joint  Test  Director 

JTF 

JADS  Joint  Test  Force 

LAN 

Local  Area  Network 

LFP 

Live  Fly  Phase 

LOS 

Line-of-Sight 

LSP 

Linked  Simulators  Phase 

LTE 

Launch-to-Eject 

LTP 

Laboratory  Test  Plan 

MCMTOMF 

Mean  Corrective  Maintenance  Time  for  Operational  Mission  Failures 

MNS-1 

MIL-STD-1553  Networking  System 

MOA 

Memorandum  of  Agreement 

MTBOMF 

Mean  Time  Between  Operational  Mission  Failures 

MWS 

Missile  Warning  System 

NAWCWPNS 

Naval  Air  Warfare  Center  Weapons  Division 

NIU 

Network  Interface  Unit 

NRNet 

NAWCWPNS  Real-Time  Network 

OAR 

Open  Air  Range 

OD 

Operations  Directive 

OFP 

Operational  FlightProgram 

OFS 

Operational  Flight  Software 

OPSEC 

Operations  Security 

OR 

Operation  Requirements 

OSD 

Office  of  the  Secretary  of  Defense 

OT 

Operational  Testing 

OT&E 

Operational  Test  and  Evaluation 

PDU 

Protocol  Data  Unit 

PI 

Program  Introduction 

PTP 

Program  Test  Plan 
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RAM 

Reliability,  Availability,  Maintainability 

RDT&E 

Research,  Development,  Test,  and  Evaluation 

RSS 

Root-Sum-Square 

RT 

Remote  Terminal 

SA 

Subaddress 

SGI 

Silicon  Graphics,  Inc. 

SIMLAB 

Sidewinder  Simulation  Laboratory 

SIT 

System  Integration  Test 

SIU 

Simulation  Interface  Unit 

SMS 

Stores  Management  System 

SNAP 

Simulator  Network  Analysis  Project 

SNMP 

Simple  Network  Management  Protocol 

SOC 

Statement  of  Capability 

ssc 

Simulation  Specific  Component 

STRICOM 

Simulation,  Training,  and  Instrumentation  Command 

SUT 

System-Under-Test 

s/w 

Software 

T&E 

Test  and  Evaluation 

TAP 

Test  Activity  Plan 

TCAC 

Test  Control  and  Analysis  Center 

TCP/IP 

Transmission  Control  Protocol/Intemet  Protocol 

TEMP 

Test  and  Evaluation  Master  Plan 

TFR 

Test  Facilities  and  Resources 

TM 

Telemetry 

TOF 

Time  of  Flight 

TSPI 

Time-Space-Position  Information 

UDS 

Universal  Documentation  System 

UDX 

Universal  Data  Exchange 

UTC 

Universal  Time  Code 

v&v 

Verification  and  Validation 

VTC 

Video  Teleconference 

W&A 

Verification,  Validation,  and  Accreditation 

WAN 

Wide  Area  Network 

WRAs 

Weapon  Replacement  Assemblies 

WSIC 

Weapons  System  Integration  Center 

WSSF 

Weapon  System  Support  Facility 
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